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ABSTRACT 

Many  loops  can  be  more  easily  understood  and  manipulated  if  they  arc  viewed  as  being 
built  up  out  ol  operations  on  sequences  of  values.  A  notation  is  introduced  which  makes  this 
viewpoint  explicit  Using  it  loops  can  be  represented  as  compositions  of  functions  operating 
on  sequences  of  values.  A  library  of  standard  sequence  functions  is  provided  along  with 
facilities  for  defining  additional  ones. 

The  notation  is  not  intended  to  be  applicable  to  every  kind  of  loop.  Rather,  it  has  been 
simplified  wherever  possible  so  that  straightforward  loops  can  be  represented  extremely 
easily.  The  expressional  form  of  the  notation  makes  it  possible  to  construct  and  modify  such 
loops  ntpidly  and  accurately.  The  implementation  of  the  notation  docs  not  actually  use 
sequences  but  rather  compiles  loop  expressions  into  iterative  loop  code.  As  a  result,  using  the 
notation  docs  not  lead  to  a  reduction  in  run  time  efficiency. 


This  report  describes  research  done  at  the  Artificial  Intelligence  laboratory  of  the  Massachusetts  Institute  of 
Technology.  Support  for  the  laboratory's  artificial  intelligence  research  has  been  provided  in  part  by  the 
Advanced  Research  Projects  Agency  of  the  Department  of  Defense  under  Office  of  Naval  Research  contract 
NOOl) 14-800-0505.  and  in  part  by  National  Science  Foundation  grant  MCS-7912179. 


'Hie  views  and  conclusions  contained  in  this  paper  arc  those  of  the  author,  and  should  not  be  interpreted  as 
necessarily  representing  the  official  policies,  either  expressed  or  implied,  of  the  Department  of  Defense,  or  the 
United  Suites  Government. 
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Introduction 

This  paper  presents  an  cxprcssion.il  loop  notation  based  on  the  ideas  described  in  fl6,17|.  lire  notation 
makes  it  possible  to  represent  loops  as  compositions  of  functions  applied  to  sequences  of  values,  'lire 
principal  benefit  of  the  notation  is  that  it  brings  the  powerful  metaphor  of  expressions  and  deeomposability  to 
bear  on  the  domain  of  loops.  Wherever  this  metaphor  can  be  applied,  it  makes  algorithms  much  easier  to 
construct,  understand,  and  modify. 

The  paper  is  divided  into  four  parts.  The  first  pari  discusses  what  it  means  to  view  a  limp  as  an  expression 
composed  of  functions  operating  on  sequences  of  values.  It  then  presents  the  major  features  of  the  notation 
in  terms  of  the  exprcssion.il  metaphor.  It  concludes  by  discussing  the  key  places  where  the  notation  docs  not 
completely  support  the  cxprcssional  metaphor. 

The  implementation  of  the  notation  docs  not  support  sequences  as  act  ual  data  objects,  but  rather  compiles 
loop  expressions  into  iterative  loops  which  operate  on  sequences  one  element  at  a  time.  Hie  second  section  of 
the  paper  presents  a  number  of  additional  features  of  the  notation  which  arc  best  understood  from  the  point 
of  view  of  this  element  at  a  time  perspective.  Iliis  part  of  the  paper  concludes  with  a  large  example  which 
shows  the  way  the  notation  is  intended  to  be  used. 

The  third  section  of  the  paper  evaluates  the  notation  from  several  points  of  view.  First  the  limits  of  the 
applicability  of  the  notation  arc  described  in  detail.  The  notation  is  not  intended  to  be  applicable  to  every 
kind  of  loop.  Rather,  it  is  designed  to  make  it  particularly  easy  to  represent  and  manipulate  the  kind  of 
straightforward  loops  which  appear  most  commonly  in  programs,  lly  focusing  on  the  main  concept  and 
resisting  the  temptation  to  add  embellishments,  the  notation  is  rendered  semantically  dean  and  easy  to 
understand. 

Second,  the  efficiency  of  the  code  produced  for  loop  expressions  is  discussed.  Due  to  the  fact  that  the 
notation  can  be  directly  compiled  into  iterative  loop  code,  there  is  no  need  to  suffer  the  kind  of  efficiency 
penalties  which  would  be  associated  with  actually  implementing  the  notation  in  terms  of  data  objects 
representing  sequences.  Appendix  A  contains  an  in  depth  description  of  the  compilation  process. 

Third,  it  is  argued  that  the  notation  could  be  implemented  as  a  logical  extension  to  almost  any  language. 
The  notation  has  already  been  implemented  as  a  LispMachinc(18J/MacLispf9J  macro  package  LETS 
("let  ess").  (Note  that  several  of  the  macros  described  in  this  paper  end  in  the  letter  "S’*,  This  "S"  stands  for 
"sequence",  and  in  all  eases  it  is  pronounced  separately.)  This  paper  discusses  the  notation  in  the  context  of 
this  particular  implementation  and  the  examples  arc  all  couched  in  terms  of  I  jsp.  However,  none  of  the  basic 
concepts  behind  the  notation  have  anything  to  do  with  the  Lisp  language  per  se.  Introducing  the  cxprcssional 
notation  as  an  extension  to  the  language  Ada  [1]  is  discussed. 

The  fourth  and  final  part  of  the  paper  presents  a  comprehensive  comparison  between  the  cxprcssional 
notation  and  other  looping  constructs.  ITic  concept  of  cxprcssional  loops  presented  here  was  motivated  by 
observing  regularities  in  the  kinds  of  straightforward  loops  which  appear  in  programs  most  often  (16].  Over 
the  years,  many  language  designers  have  also  noticed  various  aspects  of  these  regularities  and  therefore  many 
of  the  key  features  of  the  cxprcssional  notation  appear  in  one  form  or  another  in  currently  existing  looping 
constructs.  The  constructs  which  are  most  similar  appear  in  die  languages  API.  (10],  Hibol  [13),  and 
Model  [11],  The  advantage  of  the  notation  presented  here  is  that  it  distills  these  concepts  into  a  semantically 
complete  whole  which  is  easy  to  understand,  easy  to  compile,  and  easy  to  add  as  an  extension  to  current 
languages. 
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I  *  The  Expressional  Metaphor 

The  key  property  of  expressions  which  makes  (hem  particularly  easy  to  construct,  manipulate,  and 
understand  is  dccomposability.  Given  an  expression,  it  is  easy  to  decompose  it  inu>  separate  parts  each  of 
which  (in  the  absence  of  sidc-cfTccts)  can  be  completely  understood  in  isolation  from  all  of  the  other  parts. 
Further,  the  behavior  of  the  expression  as  a  wlwlc  is  merely  the  composition  of  the  behaviors  of  its  parts. 

Consider  the  expression  "(SIN  (SORT  X))".  Its  two  parts  can  be  understood  in  isolation.  For  example, 
you  can  understand  what  the  SQRT  docs  (i.c„  compute  the  square  root  of  its  input)  without  having  to  think 
about  where  its  input  comes  from,  where  its  output  will  be  used,  or  about  anything  else  that  is  going  on  in  the 
expression.  The  only  interaction  between  the  two  functions  is  the  data  flow  between  them.  In  order  to 
understand  what  the  expression  as  a  whole  docs,  (i.c.,  compute  the  sine  of  the  square  root  of  its  input)  you 
merely  have  to  compose  your  understandings  of  the  two  functions. 

The  primary  goal  behind  the  design  of  the  loop  notation  presented  here  has  been  the  development  of  a 
notation  which  has  die  property  of  dccomposability. 

Viewing  Loops  as  Expressions  Involving  Sequences 

In  order  to  represent  loops  as  expressions,  the  concepts  of  sequences  and  sequence  functions  which  operate 
on  them  arc  introduced.  In  this  context,  all  other  data  structures  arc  referred  to  as  unitary.  A  sequence  is  an 
ordered  (possibly  infinite)  one  dimensional  series  of  unitary  data  objects.  A  sequence  function  is  a  function 
which  produces  one  or  more  sequences  as  outputs  and/or  consumes  one  or  more  sequences  as  inputs.  I^oops 
arc  represented  as  expressions  built  out  of  sequence  function  applications. 

For  reasons  of  efficiency,  sequences  arc  not  represented  as  actual  data  structures  at  run  time.  Rather, 
expressions  involving  sequences  arc  compiled  into  iterative  loops  in  which  the  existence  of  the  sequences  is 
only  implicit.  This  is  analogous  to  the  way  in  which  many  program  constructs  arc  handled  by  compilers.  For 
example,  references  to  components  of  a  record  structure  in  a  program  typically  appear  to  pass  indirectly 
through  the  structure  as  a  whole.  However,  for  efficiency,  such  references  arc  generally  compiled  into  direct 
accesses  on  the  components  as  if  they  were ‘atomic  objects.  The  existence  of  the  structure  as  an  identifiable 
unit  is  only  implicit  in  the  compiled  code. 

Sequences  and  sequence  functions  exist  as  explanatory  devices.  Ihe  point  is  that  thinking  of  loops  as 
compositions  of  functions  operating  on  sequences  makes  them  easier  to  understand.  The  fact  that  the 
compiled  form  is  very  different  is  in  general  of  no  import.  (The  second  part  of  this  paper  discusses  situations 
where  the  user  docs  have  to  be  cognizant  of  die  compiled  form.) 

Consider  the  program  SUM- POSITIVE -EXPRESSIONAL  below.  Its  body  is  a  sequence  expression  which 
sums  up  the  positive  elements  of  a  one  dimensional  array.  Given  an  array  containing  <0  1-12  -2>  the 
program  would  produce  the  result  3. 

(defun  aum-posltlve-expresstonal  (vector) 

(Rsum  (Fgreater  (Evector  vector)))) 

The  sequence  function  EVECTOR  ("ce  vector *)  takes  in  a  one  dimensional  array  and  enumerates  a  sequence 
of  the  data  items  in  the  array  (c.g„  producing  the  sequence  [0  1-12  -2]).  (Note  that  the  names  of  the 
built-in  sequence  functions  all  begin  with  prefix  letters.  These  letters  indicate  the  type  of  operation 
performed  by  the  sequence  function.  Ihe  letter  "£"  stands  for  enumerate,  "G"  stands  for  generator,  "F" 
stands  for  filter,  and  "R"  stands  for  reduce.  In  each  ease,  these  prefix  letters  are  pronounced  separately.) 

Ihe  sequence  function  FGREATER  ("ef greater")  takes  in  a  sequence  and  filters  it  producing  a  sequence 
containing  only  the  positive  elements  in  the  input  sequence  (c.g.,  producing  [_  1  _  2  _]).  Note  that  the 
action  of  the  filter  is  encoded  by  leaving  some  of  the  slots  in  the  output  sequence  empty  (symbolized  by  V) 
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rather  than  by  creating  a  sequence  of  reduced  length.  In  order  to  make  this  work,  everything  is  set  up  so  that 
empty  slots  arc  ignored  in  subsequent  computations.  I  he  reason  why  the  concept  of  empty  slots  is  useful 
stems  from  the  element  at  a  lime  metaphor  and  will  be  discussed  in  the  second  part  of  this  paper. 

I  lie  sequence  function  RSUM  {"ursuni")  takes  in  a  sequence  of  integers  and  reduces  it  to  a  unitary  object 
containing  their  sum  (c.g..  3).  Ihc  sequence  expression  above  is  easy  to  understand  because  the  actions  of  the 
sequence  functions  can  be  understood  in  isolation  from  each  other,  and  the  action  of  the  expression  ns  a 
whole  (i.e.,  to  sum  the  positive  elements  of  a  vector)  is  simply  the  composition  of  these  actions.  Further,  it  is 
as  easy  to  mttdify  as  any  other  expression. 

Simple  Examples  of  Sequence  Functions 

This  section  presents  a  number  of  built-in  sequence  Junctions  which  arc  used  in  examples  in  the  rest  of 
this  paper.  Hie  complete  set  of  built-in  sequence  functions  provided  as  part  of  the  LETS  macro  package  is 
presented  in  Appendix  B.  'Ihcrc  arc  three  basic  kinds  of  sequence  functions:  unitary^sequence, 
sequencer-unitary,  and  sequence* sequence.  ’Ihc  most  common  kind  of  umtary+scqucncc  function  takes  some 
aggregate  data  object  and  creates  a  sequence  of  its  components. 

Eilstftr 

'lakes  in  a  list  and  creates  a  sequence  of  its  elements. 
c.g.,  (El  1st  ’(12  3))  *>  [1  2  3] 

Esubl  lets  list 

'fakes  in  a  list  and  creates  a  sequence  of  its  successive  sublists. 
c.g..  (Esublists  ’(1  2  3))  »>  [(12  3)  (2  3)  (3)3 

Evector  vector  &opt1onal  [first  Q)  [last [l-  (array-length  vector))) 

l  akes  in  a  one  dimensional  array  and  creates  a  sequence  of  its  elements. 
c.g..  (Evector  <1  2  3>)  ■>  [1  2  3] 

Eflle  filename 

Creates  a  sequence  of  values  by  reading  all  of  the  objects  out  of  the  file. 
c.g., (Eflle  "data. lisp")  ■>  [12  3] 

If  the  file  "data. lisp"  contains  "12  3" 

Another  family  of  unitary^sequence  functions  computes  a  sequence  of  values  according  to  Some  foimula. 
Erange  first  last  Aoptlonal  [step-size  1) 

Creates  a  sequence  of  Integers  by  counting  from  first  to  last  by  the  positive  increment  step-size. 

C.g.,  (Erange  4  8  2)  «>[4  8B] 

(sequence  object 

Generates  an  infinite  sequence  all  of  whose  elements  arc  object. 
c.g„  ((sequence  ’A)  »>  [A  AA  . ..] 

'Ihc  most  common  kind  of  sequence*unitar y  function  takes  in  a  sequence  and  combines  the  elements  in  it 
together  into  an  aggregate  data  structure. 
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R1  1st  sequence 

CONScs  the  non-empty  values  in  a  sequence  into  a  list 
C.g.,  (R11st[12_3])*>(12  3) 

Rvactor  vector  sequence  top  1 1  on  al  [first  0)  (last  ( 1-  ( array- length  vector) )) 

Stores  the  non-empty  values  in  a  sequence  into  successive  slots  of  a  one  dimensional  array. 

C.g..  (Rvector  <A  B  C  D>  [1  2  _  3])  «>  <1  2  3  D> 

Rf  11a  filename  sequence 

Writes  the  non-empty  values  in  a  sequence  into  the  indicated  file. 
c.g..  (Rf  11a -data. lisp"  [12-3])  «>T 

"<cr>l  <cr>2  <cr>3  "  Is  printed  In  "data. lisp" 

Another  kind  of  sequence+unitary  function  computes  some  summary  value  based  on  the  values  in  die 
sequence. 

Rsum  sequcnce-of  integers 

Computes  the  sum  of  the  non-empty  integer  values  m  a  sequence. 

C.g.,  (Rsum  [1  2_3])  ■>  8 

Rcount  sequence 

Counts  the  number  of  non-empty  items  in  a  sequence. 
c.g.,  (Rcount  [A  C])  «>  3 

Rlast  soyumr  Aoptlonal  (default H XL) 

Returns  the  last  non-empty  element  (if  any)  of  the  sequence  as  its  value;  otherwise  returns  default. 

C.g..  (Rlast[ABC_])OC 

Scqucncc+sequcncc  (unctions  take  in  a  sequence  of  values  and  ampule  some  related  sequence.  They  tend 
to  be  much  more  idiosyncratic  than  other  kinds  Of  sequence  functions  and  only  one  is  predefined.  The  next 
section  describes,  among  other  things,  the  mechanisms  which  arc  used  to  create  user  defined 
scquencc+sequence  functions. 

Fgraatar  sequence"  of- numbers  fcop  1 1  o  n  a  1  (limit  0) 

Selects  the  non-empty  elements  of  a  sequence  of  integers  greater  than  limit. 
c.g.,  (Fgraatar  [12.3]  2)  «>  [ _ 3] 

The  programs  below  give  a  number  of  examples  of  loops  built  up  out  of  the  sequence  functions  described 
above.  COPY-LIST  copies  a  list  by  enumerating  the  items  in  the  list  and  then  CONSing  diem  up  into  a  new  list 
LAST  enumerates  ail  of  the  sublists  in  a  list  and  then  returns  the  last  one.  SUM-FIRST-N  adds  up  the  first  N 
integers  by  enumerating  the  integers  and  then  summing  this  sequence  of  values. 

(defun  copy- list  (Hat) 

(Rllst  (Eilat  Hat))) 

(defun  last  (Hat) 

(Rlast  (Esubllsts  Hat))) 

(defun  sum-flrst-n  (n) 

(Rsum  (Erange  1  n))) 
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FILE-LENGTy  computes  the  number  of  records  in  a  file  by  enumerating  them  and  then  counting  the  items 
in  this  sequence.  DUMP-VECTOR  prints  the  elements  of  a  vector  into  a  file.  ZERO-VECIOR  initializes  a  vector 
by  setting  the  elements  to  zero.  It  uses  GSEQUENCE  in  order  to  generate  a  sequence  of  zeros  to  use. 

(defun  file-length  (file-name) 

(Rcount  (Efile  file-name))) 

(defun  dump-vector  (file-name  vector) 

(Rfile  file-name  (Evector  vector))) 

(defun  zero-vector  (vector) 

(Rvector  vector  (Gsequence  0))) 


Meta  Sequence  Functions 

In  addition  to  predefined  sequence  functions,  the  LETS  macro  package  supports  several  meta  sequence 
/unctions  which  make  it  easy  for  the  user  to  create  new  sequence  functions.  'Hie  basic  action  of  a  meta 
sequence  function  is  to  take  an  ordinary  function  and  convert  it  into  a  function  on  sequences.  Hach  meta 
sequence  function  builds  a  particular  kind  of  sequence  function. 

'Hie  most  basic  meta  sequence  function  is  (MAPS  function  sequence ...).  MAPS  is  a  generalization  of  the 
Lisp  function  MAP  and  is  the  principal  method  for  creating  user  specified  scqucncc+scquaicc  functions.  It 
takes  Junction  and  converts  it  into  a  sequence  function  which  takes  in  the  sequence  inputs  and  creates  a 
sequence  output.  The  number  of  sequences  provided  must  be  compatible  with  the  number  of  arguments 
required  by  Junction.  The  nth  element  of  the  output  sequence  is  computed  by  applying  function  to  the  nth 
elements  of  the  input  sequences.  However,  if  the  nth  element  of  any  of  the  input  sequences  is  empty  then 
Junction  is  not  applied  and  the  nth  element  of  the  output  is  empty.  Note  that  the  length  of  the  output 
sequence  is  the  same  as  the  length  of  the  shortest  input  sequence.  Ihc  Junction  parameter  can  be  cither  a 
quoted  function  name,  or  a  quoted  LAMBDA  expression  (or  a  macro  that  expands  into  either  one).  For 
example,  the  program  PAIRWISE-MAX  takes  in  two  lists  and  creates  a  list  where  each  element  is  the  maximum 
of  the  corresponding  elements  in  the  two  input  lists.  The  program  SQUARE- LIST  creates  a  list  of  the  squares 
of  the  items  in  a  list 

(defun  pairwise-max  (llstl  11st2) 

(Rllst  (maps  #’max  (El  1st  llstl)  (Ellst  11st2)))) 

(defun  square-list  (list) 

(Rllst  (maps  #'(1ambda  (x)  (*  x  x))  (Ellst  list)))) 

The  program  TIMES- N  multiplies  every  element  in  a  list  by  a  parameter  N.  The  point  of  this  example  is 
that  the  functional  argument  to  MAPS  (and  the  functional  arguments  to  the  other  mem  sequence  functions 
described  below)  can  refer  to  any  number  of  free  variables.  These  free  variables  do  not  have  to  be  declared 
special  because  the  LETS  macro  package  renders  the  loop  entirely  .as  inline  code. 

(dafun  tlmes-n  (list  n) 

(Rllst  (maps  ^’(lambda  (x)  (•  x  n))  (Ellst  list)))) 

An  extended  form  of  MAPS  is  the  meta  sequence  function  ( SCANS  Junction  init  sequence ... ) .  This  creates  a 
sequence  function  with  an  internal  state  variable.  The  input  function  must  be  a  function  of  n+ 1  arguments 
where  n  is  the  number  of  sequences  supplied.  The  elements  of  the  output  arc  the  successive  values  of  the 
state  not  including  its  initial  (unitary)  value  init.  The  nlh  value  of  the  state  is  computed  by  calling  ^function 
with  the  prior  value  of  the  state  as  its  first  argument  and  the  nth  elements  of  the  inputs  as  its  remaining 
arguments.  However,  if  the  nth  element  of  any  of  the  input  sequences  is  empty  then  Junction  is  not  applied. 
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the  state  is  not  changed,  and  the  nth  element  of  the  output  is  empty.  As  with  MAPS,  the  length  of  the  output 
sequence  is  the  same  as  the  length  of  the  shortest  input  sequence.  SCANS  is  useful  for  creating  a  sequence 
function  corresponding  to  a  recurrence  relation.  For  example,  the  program  SQUARES  computes  a  list  of  the 
first  N  squares  without  doing  any  multiplication  by  taking  advantage  of  fact  that  n 2  *  { n-l )  2+2n-l. 

(defun  squares  (n) 

(Rllst  (scans  #'( lambda  (n-l-squared  n)  (+  n-l-squared  n  n  *1)) 

0  (Erange  In)))) 

The  meta  sequence  function  (FILTERS  function  sequence ... )  is  used  to  create  sequence  functions  like 
FGREATER  which  select  a  subsequence  of  a  sequence.  ‘IT»c  elements  of  the  output  sequence  arc  computed  as 
follows.  I  f  the  result  of  applying  function  to  the  nth  elements  of  the  input  sequences  is  non-N  I L  then  the  nth 
element  of  the  first  input  is  used  as  die  nth  ctcmcnt  of  the  output;  otherwise  the  nth  output  element  is  empty. 
As  with  MAPS,  if  the  nth  element  of  any  of  the  input  sequences  is  empty  .then  function  is  not  applied  and  the 
nth  element  of  the  output  is  empty. 

As  an  example,  consider  the  function  SUM-POSITIVE-FILTER.  It  uses  the  meta  sequence  (Unction 
FILTERS  instead  of  the  sequence  function  FGREATER.  The  function  REMQ  takes  in  a  list  and  CONScs  up  a  new 
list  which  is  the  same  except  that  all  instances  of  a  given  item  are  removed.  It  uses  a  filter  to  select  which  list 
elements  to  keep. 

(defun  sum-posltlve-fllter  (vector) 

(Rsuni  (filters  #'p1usp  (Evector  vector)))) 

(defun  remq  (Item  list) 

(Rllst  (filters  #' (Tamtoda  (x)  (not  (oq  x  Item)))  (Eilat  list)))) 

User  specified  sequence+unitary  functions  can  be  created  by  using  the  meta  sequence  (Unction 
(REDUCES  function  init sequence...).  This  creates  a  sequence  (Unction  with  an  internal  stale  variable.  The 
state  is  initialized  to  the  (unitary)  value  init.  The  nth  value  of  the  state  is  computed  by  calling  Junction  with 
the  prior  value  of  the  state  as  its  first  argument  and  the  nth  elements  of  the  inputs  as  its  remaining  arguments. 
However,  if  the  nth  element  of  any  of  die  input  sequences  is  empty  then  function  is  not  applied  and  the  state 
is  not  changed.  When  the  input  sequences  ait  exhausted,  the  final  value  of  the  state  variable  is  returned  as 
the  (unitary)  result  If  there  arc  no  non-empty  elements  in  die  input  sequences  then  the  value  init  will  be 
returned.  Ihc  meta  sequence  (Unctions  REDUCES  and  SCANS  arc  very  closely  related.  The  expression 
(REDUCES  function  init  sequence)  is  the  same  as  (RLAST  ( SCARS  function  init  sequence)  init). 

As  examples,  consider  die  following  two  (Unctions.  SUM-POSITIVE-REDUCER  uses  REDUCES  instead  of  a 
call  on  RSUM.  make-set  takes  in  a  list  possibly  containing  duplicate  elements  and  creates  a  list  without  any 
duplicates  which  contains  die  same  elements.  The  key  pmblcm  is  removing  duplicates.  To  do  this,  the 
function  uses  a  reducer  which  adds  die  current  element  into  the  list  being  created  only  if  it  is  not  a  member  of 
the  list  already. 
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(defun  sum-posltlve-redu^er  (vector) 

(reduces  #’+  0  (Fgreater  (Evector  vector)))) 

(defun  make-set  (list) 

(reduces  #‘(1ambda  (state  Item) 

(cond  ((member  Item  state)  state) 

(T  (cons  Item  state)))) 
nil 

(El  1st  list))) 

'Ihc  most  basic  way  to  create  a  unitary+sequencc  function  is  to  use  the  meta  sequence  function 
(GENERATES  function  inii).  The  sequence  function  produced  creates  a  sequence  of  elements  where  the 
(unitary)  value  mil  is  the  first  clement  and  where  each  successive  clement  is  computed  from  the  prior  clement 
by  evaluating  function  with  the  prior  clement  as  its  argument.  Note  that  the  output  sequence  is  infinite  in 
extent.  Ihc  next  two  meta  sequence  functions  can  be  used  to  create  finite  sequences. 

A  loop  expression  which  contains  only  a  generator  will  never  terminate  because  it  operates  on  an  infinite 
sequence.  However,  if  a  loop  expression  is  working  on  several  sequences  some  of  which  are  finite  and  some 
of  which  arc  not,  it  will  terminate  as  soon  as  the  shortest  finite  sequence  has  been  exhausted.  This  is  discussed 
further  in  the  section  on  termination  below. 

Generators  arc  typically  used  in  loop  expressions  in  conjunction  with  finite  sequences  of  unknown  length. 
For  example,  the  program  DIGITS- TO-NUMBER  takes  in  a  list  of  one  digit  numbers  and  computes  the 
corresponding  integer  (c.g.,  ’  (1  2  3)  becomes  123).  'Jhe  loop  expression  works  with  two  basic  sequences. 
It  enumerates  the  digits  in  the  list  in  reverse  order  (i.c.,  least  significant  digit  first).  It  also  creates  an 
unbounded  sequence  of  scale  factors  consisting  of  the  successive  powers  of  ten.  Ihc  result  is  computed  by 
summing  up  the  product  of  each  digit  with  its  corresponding  scale  factor.  Ihc  loop  terminates  when  the  digits 
run  out. 

(defun  dlglts-to-number  (digit-list) 

(Rsum  (maps.  #’•  (El  1st  (reverse  digit-list)) 

(generates  #'(lambda  (x)  (•  x  10))  1)))) 

Another  example  is  the  program  F  ILL-VECTOR  which  takes  in  a  fist  and  uses  it  to  initialize  the  elements  of 
a  vector.  If  there  arc  more  elements  in  the  list  than  in  the  vector,  the  extra  list  elements  arc  ignored.  On  the 
other  hand,  if  there  arc  more  elements  in  the  vector,  the  last  clement  of  the  fist  is  used  to  fill  out  all  of  the 
remaining  elements  of  the  vector. 

(defun  fill-vector  (vector  list) 

(Rvector  vector  (maps  #'car  (generates  #'(lambda  (x)  (or  (edr  x)  x) )  list)))) 

Note  that  it  is  the  size  of  the  vector  which  controls  the  computation,  not  the  length  of  the  fist  To  do  this 
conveniently,  a  generator  is  created  which  generates  the  successive  sublists  of  the  list,  but  which  continues  to 
generate  the  List  sublist  indefinitely  once  it  has  been  reached,  lhe  function  CAR  is  MAPScd  over  the  generated 
sequence  in  order  to  get  the  desired  list  elements.  These  elements  arc  then  stored  in  the  vector.  RVECTOR 
contains  a  termination  which  stops  the  loop  when  the  vector  has  been  filled  up. 

The  meta  sequence  function  (TRUNCATES./wmf/'on sequence...)  is  used  to  create  sequence  functions 
which  take  in  potentially  infinite  sequences  and  return  sequences  which  have  been  truncated  to  finite  length. 
The  function  argument  is  applied  to  successive  groups  of  corresponding  elements  of  the  input  sequences.  The 
output  sequence  is  composed  of  the  elements  of  the  first  input  sequence  up  to  but  not  including  the  first 
element  corresponding  to  a  non-NIL  evaluation  of  function.  « u,'th  th-'  ..ic*r  meta  sequence  functions,  if  any 
of  the  nth  elements  of  the  input  sequences  are  empty  .then  fun  i  is  nu  applied  and  the  nth  output  clement 
is  empty.  Note  that  the  output  sequence  is  typically  shorter  than  any  of  the  input  sequences,  and  can  be  of 
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length  zero. 

Consider  the  following  two  examples  which  arc  the  same  as  programs  presented  in  the  last  section  except 
that  they  use  explicit  generators  and  truncators.  SUM- FIRST -N-TRUNCATOR  generates  an  infinite  sequence  of 
integers  and  truncates  it  alter  N.  The  resulting  sequence  is  then  summed.  LAST-TRUNCATOR  generates  the 
successive  CDRs  of  a  list  and  truncates  this  when  NIL  is  reached.  The  last  suhlist  is  returned. 

(defun  sum-f Irst-n-truncator  (n) 

(Rsum  (truncates  #’(lambda  (x)  (>  x  n))  (generates  #'1+  1)))) 

(defun  tast-truncator  (list) 

(Rlast  (truncates  #*null  (generates  #'cdr  list)))) 

Truncators  arc  a  great  deal  like  filters  in  that  they  take  in  a  sequence  and  return  a  restricted  sequence. 
However,  they  differ  in  one  very  important  way  --  they  cause  the  loop  to  terminate.  Uven  when  a  filler  selects 
only  a  finite  number  of  elements  out  of  an  infinite  sequence,  it  never  causes  the  loop  to  terminate.  For 
example,  the  program  SUM-F IRST-N-BUGGY  will  never  tciminatc  even  though  the  correct  numbers  have  been 
selected.  Note  that  it  is  not  in  general  possible  to  detect  when  a  filter  has  reached  a  point  where  no  more 
elements  of  die  input  sequence  will  satisfy  the  filter  test. 

(defun  sum-f 1rst-n-buggy  (n) 

(Rsum  (filters  #'(lambda  (x)  (not  (>  x  n)))  (generates  #'1+  1)))) 

TTie  meta  sequence  function  ( ENUMERATES  truncate-function generatc-function  init )  is  an  abbreviation  for 
the  common  combination  ( TRUNCATES  truncate-junction  (GENERATES  gcncrate-function  init)).  For  example 
the  two  functions  above  could  be  written  more  compactly  as  follows: 

(defun  sum-flrst-n-enumerator  (n) 

(Rsum  (enumerates  #’ (lambda  (x)  (>  x  «))  #'1+  l))) 

(defun  last-enumerator  (list) 

(Rlast  (enumerates  #'nu11  #’cdr  list))) 

The  meta  sequence  functions  arc  an  essential  part  of  the  cxprcssional  loop  notation  because  they  provide  a 
convenient  mechanism  wh*  ,.>y  the  user  can  create  additional  operations  on  sequences. 

Lets 

In  an  ordinary  expression,  if  you  want  to  use  the  value  of  a  subexpression  in  two  places,  you  have  to  bind 
this  value  to  a  variable.  Ihe  prototypical  way  to  do  this  in  I.isp  is  with  the  macro  LET.  TTie  macro  LETS  fills 
the  identical  role  in  sequence  expressions.  In  the  absence  of  side-cficcts.  the  only  effect  of  using  LET  or  LETS, 
rather  than  merely  duplicating  die  subexpression,  is  increased  efficiency  due  to  executing  die  subexpression 
only  once  and  a  potential  gain  in  the  clarity  of  die  expression. 

The  macro  LETS  is  analogous  to  dcstructuring  LET*.  It  has  a  list  of  bound  variable  value  pairs  which  arc 
executed  sequentially  so  dwt  you  can  use  a  variable  in  the  compulation  of  the  value  to  be  bound  to  a  later 
variable.  Instead  of  a  single  variable,  a  tree  of  variables  can  be  used  to  specify  dcstructuring.  Alternately,  the 
value  can  be  omitted  in  which  ease  it  is  assumed  that  there  is  no  initial  value  at  all.  In  this  ease  the  variable 
must  be  set  before  it  can  be  read.  ITicsc  three  eases  arc  illustrated  in  the  example  below. 

(lets  ( ( v 1  valuel)  ( ( v2  v3 )  value2)  v4) 

.  body) 

Inside  die  body  of  a  LETS  you  can  use  die  form  (SETQ  variable  value)  in  order  to  assign  a  sequence  value 
to  a  sequence  variable.  The  initializing  values  arc  handled  as  if  they  were  sequentially  assigned  to  the  bound 
variables  inside  the  LETS  as  illustrated  below.  Dcstructuring  is  implemented  in  terms  or  the  appropriate  CAR 
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(lets  (vl  x  v2  v3  v4) 

(setq  vl  valuel) 

(setq  x  value2) 

(setq  v2  (mapS  0’car  x)) 

(setq  v3  (maps  jP’cadr  x )) 

.  body) 

I-uich  initializing  value  must  be  a  sequence.  LETS  cannot  be  used  to  bind  a  variable  to  a  unitary  value, 
owever,  using  GSEQUENCE,  you  can  bind  a  sequence  variable  to  an  infinite  sequence  of  a  unitary  value, 
lich  will  usually  be  sufficient.) 

'Ihc  macro  LETS  contains  a  body  which  consists  of  one  or  more  loop  expressions.  ’Ihcsc  expressions  can 
cr  to  the  sequences  bound  to  the  sequence  variables,  and  can  result  in  either  sequence  or  unitary  values, 
c  value  of  the  last  form  must  be  unitary  and  is  returned  as  the  result  of  the  LETS  as  a  whole.  (This  and 
nc  additional  aspects  of  LETS  stent  from  the  element  at  a  time  metaphor  and  will  not  be  discussed  in  detail 
til  the  next  part  of  the  paper.) 

As  a  simple  example.  01GITS-T0-MUMBER  could  be  rewritten  as  shown  below.  Two  sequence  variables 
;  used  to  make  the  loop  more  readable. 

(defun  diglts-to-number-letS  (digit-list) 

(lets  ((digits  (Ellst  (reverse  digit-list))) 

(scales  (generates  (lambda  (x)  (•  x  10))  1))) 

(Rsum  (maps  #'*  digits  scales)))) 

Since  each  sequence  variable  is  only  used  once  in  DIGITS-TO-NUMBER-LETS  the  function  is 
orithmically  identical  to  the  earlier  version  of  this  function  which  did  not  use  variables.  A  sequence 
iablc  can  of  course  be  referenced  more  than  once.  'Ibis  will  not  cause  the  sequence  to  be  computed  more 
n  once.  As  a  result,  SQUARE-LIST- LETS  below  is  more  efficient  than  SQUARE-LIST-REDUNDANT. 

(defun  square-llst-letS  (list) 

(lets  ((integers  (611st  list))) 

(R1 1st  (mapS  #' •  Integers  integers)))) 

(defun  square-list-redundant  (list) 

(R1 1st  (maps  #’•  (Ellst  list)  (EUst  list)))) 

The  explicit  use  of  SETQ  in  a  LETS  is  illustrated  in  the  program  DIGITS-TO-NUMBER-SETQS.  The  example 
ws  that  you  can  make  repetitive  assignments  redefining  the  value  of  a  sequence  variable  just  as  you  can  in 
ordinary  LET.  The  program  first  enumerates  the  digits.  It  then  multiplies  each  one  by  the  appropriate 
Ic  factor  and  then  sums  the  resulting  sequence.  It  is  important  to  note  that  you  cannot  use  anything  other 
n  SETQ  (or,  as  discussed  below,  MULTIPLE-VALUE)  in  order  to  assign  to  a  sequence  variable.  In  particular 
i  cannot  use  SETF  or  any  other  macro  even  if  it  expands  into  a  SETQ. 

(defun  dlglts-to-number-setq  (digits) 

(lets  (Integers) 

(setq  Integers  (Ellst  (reverse  digits))) 

(setq  Integers  (*  Integers  (generates  ^'(lambda  (x)  (•  x  10))  1))) 

(Rsum  Integers))) 

SETQs  and  other  forms  can  be  used  to  assign  to  any  number  of  free  (unitary)  variables  in  the  body  of  a 
S.  This  is  often  useful  for  passing  information  out  of  a  loop. 
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DefunS 

The  meta  sequence  functions  make  it  possible  for  a  user  to  create  his  own  sequence  operations.  However, 
these  operations  are  only  literals  and  must  be  recreated  each  time  they  .are  to  be  used.  'Hie  macro  DE  FUNS 
makes  it  possible  for  a  user  to  created  named  sequence  functions  which  he  can  then  use  in  loop  expressions. 
These  sequence  functions  are  actually  macros  which  arc  compiled  inline  by  the  LETS  macro  package. 
However,  in  the  context  of  the  cxpressional  notation,  they  are  intended  to  be  thought  of  as  functions  just  like 
any  other  function. 

(defunS  name  parameter  list 
.  body) 

The  macro  DEFUNS  is  exactly  analogous  to  DEFUN.  It  has  two  basic  parts:  a  parameter  list  and  a  body.  The 
parameter  list  is  a  list  of  variable  names  and  supports  four  keywords:  AUNITARY.  ^SEQUENCE,  ^OPTIONAL, 
and  &AUX.  Kach  of  the  keywords  is  sticky  and  specifics  the  type  of  all  of  the  parameters  which  follow  it  until 
another  keyword  changes  the  type.  The  first  two  keywords  arc  used  to  specify  whether  a  particular  parameter 
is  a  sequence  or  an  ordinary  unitary  object.  By  default,  the  parameters  arc  initially  assumed  to  be  unitary. 
Just  as  in  ordinary  l.isp.  &0PTI0NAL  specifies  that  the  following  parameters  arc  optional.  Also  just  as  in 
ordinary  Lisp,  &AUX  specifics  that  the  following  variables  are  not  parameters  at  all.  but  rather  just  internal 
values.  Oplional/initial  values  can  be  specified  using  variable  value  pairs.  However,  unlike  ordinary  Lisp,  if 
no  default  value  is  specified  then  no  value  will  be  supplied  and  the  associated  variable  must  be  set  before  it 
can  be  read. 

Ilic  body  of  a  DEFUNS  is  exactly  the  same  as  the  body  of  a  LETS  except  that  the  last  form  is  not  required  to 
yield  a  unitary  value.  Note,  however,  that  DEFUNS  is  completely  different  from  LETS  in  that  it  creates  a 
sequence  function  which  can  later  be  combined  together  with  other  sequence  functions  while  LETS  creates  an 
actual  loop.  The  value  of  the  last  form,  be  it  unitary  or  sequence,  is  returned  as  the  value  of  the  sequence 
function  being  created.  Ihc  following  examples  use  DEFUNS  in  order  to  define  a  number  of  the  standard 
sequence  functions  described  above.  Note  that  ELI  ST  returns  a  sequence  while  RSUM  returns  a  unitary'  value. 

(defunS  Ellst  (list) 

(maps  #’car  (enumerates  #’null  #‘cdr  list))) 

(defunS  Rsum  (^sequence  num) 

(reduces  0'+  0  num)) 

’Ihc  following  definitions  illustrate  the  use  of  optional  parameters.  ERANGE  Likes  an  optional  positive 
increment  which  defaults  to  one.  F GREATER  takes  an  optional  comparison  limit  which  defaults  to  zero. 
EVECTOR  takes  an  optional  interval  in  die  vector  which  defaults  to  the  full  limits  of  the  vector. 

(defunS  Erange  (first  last  Aoptlonal  (step-size  1)) 

(enumerates  #‘(lambda  (x)  (>  x  last))  ^’(lambda  (x)  (+  x  step-size))  first)) 

(defunS  Fgreater  (Asequence  integers  (unitary  Aoptlonal  (limit  0)) 

(filters  ^'(lambda  (x)  (>  x  limit))  Integers)) 

(defunS  Evector  (vector  Aoptlonal  (start  0)  (end  (l-  (array-length  vector)))) 
(mapS  #'(lambda  (x)  (aref  vector  x))  (Erange  start  end))) 

Ihc  ability  for  the  user  to  conveniently  define  his  own  named  sequence  functions  is  a  particularly 
important  part  of  the  cxpressional  loop  notation.  It  makes  it  possible  for  him  u>  extend  the  notation  to  deal 
with  the  particular  data  abstractions  he  creates.  A  detailed  example  of  this  is  given  in  a  later  section. 
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All  of  the  sequence  functions  presented  above  have  had  a  single  return  value.  The  LETS  macro  package 
supports  multiple  return  values  by  supporting  the  standard  l.ispMachine  functions  VALUES  and  MULTIPLE- 
VALUE  inside  the  body  of  a  DEFUNS  or  LETS.  For  example,  llic  function  EPLIST  takes  in  a  disembodied  plist 
and  returns  two  values:  a  sequence  of  the  property  names,  and  a  sequence  of  the  values  of  those  properties. 
VALUES  is  used  as  the  last  form  of  the  DEFUNS  in  order  to  specify  that  two  sequences  arc  being  returned. 

(defunS  Epllst  (plist  ^sequence  &aux  pointers) 

(setq  pointers  (enumerates  #’null  0’cddr  (edr  plist))) 

(values  (maps  #'c ar  pointers)  (maps  0'cadr  pointers))) 

The  program  PLIST-TO-ALIST  converts  a  plist  into  an  alist  where  the  entries  in  the  alist  arc  created  by 
CONSing  together  successive  property  value  pairs.  The  program  illustrates  how  MULTI  PLE-VALUE  can  be  used 
in  a  LETS  in  order  to  access  the  two  sequences  returned  by  EPLIST. 

(defun  plist-to-allst  (plist) 

(lets  (properties  values) 

(multiple-value  (properties  values)  (Epllst  plist)) 

(rllst  (maps  #'cons  properties  values)))) 

'Hie  function  COUNT-AND-SUM  illustrates  the  use  of  VALUES  as  the  last  form  of  a  LETS  in  order  to  specify 
that  the  loop  as  a  whole  returns  multiple  values. 

(defun  count-and-sum  (list) 

(lets  ((integers  (El  1st  list))) 

(values  (Rcount  Integers)  (Rsum  Integers)))) 

(The  MacLisp  version  of  LETS  docs  not  support  multiple  return  values  from  loops.  However,  it  does 
support  multiple  return  values  from  sequence  functions  such  as  EPLIST.) 

Where  the  Expressional  Metaphor  Breaks  Down 

There  arc  two  situations  in  which  loop  expressions  fail  to  be  faithful  to  the  expressional  metaphor.  The 
first  of  these  involves  sidc-cffccts.  If  a  sequence  Junction  performs  sidc-eflccts  which  disturb  the  actions  of 
another  sequence  function,  then  the  behavior  of  a  loop  expression  as  a  whole  can  fail  to  be  the  composition  of 
the  behaviors  of  the  sequence  functions  when  looked  at  separately.  (A  detailed  example  of  this  will  be 
discussed  later  on  in  this  paper.)  It  should  be  noted  that  die  breakdown  of  the  expressional  metaphor  in  this 
situation  is  not  at  all  surprising  considering  that  ordinary  expressions  lose  the  property  of  modularity  in  the 
presence  of  side-effects. 

ITie  second  place  where  the  expressional  metaphor  breaks  down  is  when  questions  of  termination  are 
being  considered.  As  mentioned  above,  the  termination  of  a  loop  expression  is  controlled  by  the  length  of  the 
sequences  in  it.  The  loop  expression  terminates  as  soon  as  the  shortest  sequence  in  it  is  exhausted.  This  is  an 
example  of  action  at  a  distance  which  makes  it  impossible  to  understand  the  various  parts  of  a  loop  expression 
completely  in  isolation  from  each  other. 

Consider  again  the  program  DIGITS-TO-NUMBER  (reproduced  below).  There  arc  several  questions  about 
the  sequence  functions  in  this  prugram  which  cannot  be  answered  completely  locally.  For  example,  although 
it  is  convenient  to  describe  the  generator  as  creating  an  infinite  sequence  of  powers  of  10.  it  cannot  actually  do 
that.  The  generator  will  eventually  halt  with  an  error  due  to  arithmetic  overflow  unless  some  other  sequence 
function  terminates  the  loop  before  thou.  On  the  other  hand,  in  order  to  be  sure  that  the  EUST  will  succeed 
in  enumerating  all  of  the  digits,  one  must  check  that  no  other  sequence  function  will  tenninate  the  loop 
before  the  end  of  die  list  of  digits  is  reached.  Because  of  these  problems,  you  cannot  just  compose  an 
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understanding  of  its  parts  in  order  to  understand  the  loop  expression  as  a  whole. 

(defun  dlglts-to-number  (digit-list) 

(Rsum  (maps  #'•  (El  1st  (reverse  digit-list)) 

(generates  #'( lambda  (x)  (•  x  10))  1)))) 

Fortunately,  there  is  a  middle  ground  with  regard  to  the  property  of  dccomposability.  As  discussed  in  [16]. 
as  long  as  you  make  no  statements  which  depend  on  a  specific  minimum  length  of  any  sequence,  any 
statement  which  is  true  about  a  sequence  function  in  isolation  will  be  true  when  it  is  composed  with  other 
functions  in  a  loop  expression.  For  example,  you  can  say  that  the  generator  creates  a  sequence  of  powers  of 
10  beginning  with  1.  However,  you  cannot  make  any  statement  about  whether  k  will  or  will  not  get  arithmetic 
overflow  in  the  process.  Similarly,  you  can  say  that  ELIST  enumerates  successive  elements  of  a  list  starting 
with  the  first  one.  You  can  even  say  that  it  will  produce  a  sequence  no  longer  than  the  list  However,  you 
cannot  make  any  claim  about  any  minimum  number  oflist  elements  which  definitely  will  be  enumerated. 

Given  the  kind  of  statements  you  can  dependably  make,  you  can  determine  a  great  deal  about  a  loop  by 
using  straight  composition.  For  example,  in  DIGITS-TO-NUMBER  it  is  easy  to  tell  that  the  values  in  the 
sequence  created  by  the  generator  correspond  to  successive  powers  of  10,  and  that  the  sequence  created  by  the 
ELIST  correspond  to  successive  digits  least  significant  digit  first.  In  addition,  it  is  clear  that  the  program 
multiplies  each  digit  by  the  appropriate  power  of  10  and  that  these  results  arc  summed  up. 

In  order  to  go  beyond  this  and  make  statements  about  termination,  you  must  do  more  global  reasoning. 
In  this  ease,  there  is  only  one  basic  finite  sequence  involved  (the  one  created  by  the  ELIST),  and  it  dearly 
dominates  the  computation.  As  a  result,  the  program  clearly  processes  all  of  the  digits  and  terminates 
computing  the  correct  number. 

The  two  step  reasoning  process  outlined  above  is  usually  very  satisfactory  for  the  kind  of  straightforward 
loops  the  exprcssional  notation  is  designed  to  represent  In  particular,  the  global  reasoning  about  termination 
is  usually  not  at  all  difficult 
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II  -  The  Element  ala  Time  Metaphor 

'lT>c  loop  notation  being  presented  here  supports  a  second  (element  at  a  time)  computational  metaphor  it) 
addition  to  the  cxprcssional  metaphor,  'the  cxprcssional  metaphor  is  based  on  the  idea  dial  a  sequence  is  a 
logical  unit  which  is  created  in  its  entirety  by  one  sequence  function  and  then  consumed  by  another  sequence 
function,  lhe  element  at  a  lime  metaphor  is  based  on  the  idea  that  die  computation  involving  all  of  the 
sequences  in  a  single  loop  proceeds  in  parallel  and  dial  the  loop  expression  is  essentially  describing  what 
happens  on  a  typical  step  in  this  process.  For  example,  consider  the  following  version  of  the  program 
D1GITS-T0-NUMBER. 

(defun  dlglts-to-number-elements  (digit-list) 

(lets  ((digit  (El  1st  (reverse  digit-list))) 

(scale  (generates  #'(lambda  (x)  (*  x  10))  1))) 

(Rsum  (mapS  #’*  digit  scale)))) 

In  this  program,  the  typical  value  of  DIGIT  is  an  element  of  the  sequence  created  by  enumerating  the 
input  DIGIT-LIST.  Hie  typical  value  of  SCALE  is  a  power  of  ten  taken  from  the  sequence  created  by  the 
generator.  On  each  cycle  of  the  computation,  the  value  of  DIGIT  is  multiplied  by  die  corresponding  value  of 
SCALE.  The  final  result  is  the  sum  of  these  products. 

If  you  compare  DIGITS-TO-NUMBER-ELEMENTS  with  the  program  DIGITS-TO-NUMBER-LETS  above  you 
will  see  that  die  only  actual  difference  is-  that  the  names  of  the  sequence  variables  arc  singular  instead  of 
plural,  lhe  only  important  difference  is  in  the  way  dicy  arc  described,  lhe  element  at  a  time  dcscripdon  is 
very  natural  for  some  parts  of  the  computation  (c.g..  the  multiplication  of  corresponding  values  of  DIGIT  and 
SCALE).  Other  parts  of  the  computation  (c.g,.  the  RSUM)  only  really  make  sense  from  die  point  of  view  of  the 
cxprcssional  metaphor, 

lhe  element  at  a  dmc  metaphor  is  intimately  lied  up  with  the  way  loop  expressions  arc  actually  compiled, 
’lhe  LETS  macro  package  converts  each  loop  expression  into  an  ordinary  iterative  loop.  Ihis  loop  processes 
the  sequences  in  the  expression  one  element  at  a  dme  with  each  slot  in  the  sequences  corresponding  to  one 
cycle  of  the  loop  which  is  produced,  lhe  correspondence  between  the  loop  produced  and  die  element  at  a 
time  metaphor  is  obvious.  However,  it  should  be  noted  that  while  the  compilation  process  exists  purely  as  a 
matter  of  efficiency,  the  element  at  a  dmc  metaphor  is  highlighted  in  this  discussion  because  it  is  the 
motivation  for  a  number  of  useful  facilities  supported  by  the  cxprcssional  notadon. 

The  two  metaphors  arc  really  very  separate  ideas.  One  could  easily  decide  to  support  just  one  of  them. 
For  example,  the  language  API,  supports  much  of  the  cxprcssional  metaphor  and  reladvcly  little  of  the 
element  at  a  time  metaphor,  while  the  language  Hibnl  docs  the  opposite.  Hxpcricnce  with  the  cxprcssional 
notation  suggests  that  it  is  beneficial  to  blend  these  two  ideas  together.  An  interesting  aspect  of  this  is  the  fact 
that  the  restrictions  on  the  expressiona]  metaphor  which  arc  needed  in  order  to  clearly  support  the  element  at 
a  time  metaphor  are  essentially  the  same  restrictions  which  arc  required  in  order  to  guarantee  efficient 
compilation. 
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Implicit  Maps 

The  creation  of  sequence* sequence  functions  with  MAPS  is  so  common  that  a  syntactic  sugaring  has  been 
introduced  to  make  this  easier.  Whenever  an  ordinary  unitary  Lisp  function  is  applied  to  sequences,  it  is 
implicitly  MAPSft/  over  those  sequences.  For  example,  the  following  version  of  DIGITS-TO-NUMBER  is 
equivalent  to  the  one  above. 

(defun  dlglts-to-number-lmpl Icit  (digit-list) 

(lets  ((digit  (El  1st  (reverse  digit-list))) 

(scale  (generates  #‘(1ambda  (x)  (•  x  10))  1))) 

(Rsum  (•  digit  scale)))) 

The  function  •  in  the  last  line  of  the  program  is  applied  to  two  sequence  variables  and  its  output  is  used  as 
a  sequence  by  the  RSUM.  From  the  point  of  view  of  the  cxprcssional  metaphor  this  is  a  type  conflict  and  docs 
not  make  a  great  deal  of  sense.  However,  as  a  statement  of  what  happens  to  the  typical  elements  of  the 
sequences  in  the  variables  DIGIT  and  SCALE  (i.c.,  that  they  arc  multiplied)  it  makes  a  lot  of  sense.  Implicit 
introduction  of  MAPS  is  provided  in  order  to  support  this  viewpoint 

MAPS  is  introduced  implicitly  in  situations  more  complex  than  the  one  above.  A  full  statement  of  the 
process  is  as  follows:  If  a  unitary  expression  appears  in  an  environment  where  a  sequence  value  is  expected  then 
the  entire  expression  down  to.  but  not  including,  any  cotnponents  which  create  sequences  is  separated  out  as  a 
LAMBDA  expression  and  MAPS ed.  This  is  demonstrated  by  the  following  three  pairs  of  equivalent  loop 
expressions.  The  second  pair  illustrates  the  fact  that  an  expression  might  not  refer  to  any  sequence  values  at 
all.  In  this  case  it  will  be  converted  to  a  LAMBDA  expression  of  no  arguments  and  still  be  MAPSed.  (Note  that, 
as  is.  this  particular  example  would  attempt  to  create  an  infinite  list.)  The  third  pair  illustrates  that  implicit 
MAPS  introduction  is  applied  to  expressions  involving  fexprs  and  macros  in  exactly  the  same  way  as  it  is  to 
other  expressions. 

(R1 1st  (1+  (•  (El  1st  llstl)  (♦  2  (El  1st  11st2))))) 

same  as:  (Rllst  (maps  #'(lambda  (x  y)  (1*  (•  x  (+  2  y)))) 

(El  1st  ltstl) 

(El  1st  11st2))) 

(Rllst  (neons  T)) 

same  as:  (Rllst  (maps  f'(1ambda  ()  (neons  T)))) 

(Rllst  (let  ( (z  (El  1st  list))) 

(push  (•  z  z)  stack) 

*)) 

same  as:  (Rllst  (mapS  #'( lambda  (x) 

(let  ((z  x)) 

(push  (•  z  z)  stack) 

*)) 

(El  1st  list))) 

A  potential  aspect  of  the  cxprcssional  metaphor  is  sacrificed  in  order  to  support  implicit  MAPS 
introduction.  If  nothing  else  were  said  you  would  expect  that  sequences  were  real  data  objects  and  that  they 
could  be  passed  between  non-sequence  functions  and  operated  on  by  ordinary  unitary  functions  like  CONS. 
However,  this  is  not  possible  because  all  such  expressions  arc  converted  by  implicit  MAPS  intrtxluction  into 
sequence  functions.  As  a  result,  a  sequence  can  never  be  operated  on  by  anything  other  than  a  sequence 
function  and  sequences  are  always  contained  completely  inside  sequence  expressions.  It  should  be  noted  that 
this  is  a  restriction  which  has  to  be  imposed  in  any  ease  in  order  to  insure  efficient  compilation.  If  sequences 
were  allowed  to  escape  from  the  confines  of  sequence  expressions  then  there  would  have  to  be  an  explicit 
representation  for  them,  and  it  would  not  be  possible  to  efficiently  compile  any  loop  which  communicated 
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sequences  from  (or  to)  the  surrounding  environment.  As  a  result,  nothing  of  great  import  is  actually  being 
sacrificed  in  order  to  support  implicit  MAPS  introduction. 

Filters  and  Expressions  Involving  Multiple  Sequences 

In  order  to  support  the  element  at  a  time  metaphor  every  sequence  function  (including  those  created  by 
the  meta  sequence  functions)  is  guaranteed  to  have  the  property  of  registration.  As  discussed  above,  a 
sequence  is  an  ordered  series  of  slots  containing  values.  ITie  registration  property  requires  that  the  nth 
element  in  the  sequence  produced  by  a  sequence  function  must  be  computed  from  the  nth  elements  of  the 
input  sequences  to  that  function.  The  computation  can  also  involve  suite  variables  internal  to  the  sequence 
function,  but  it  cannot  refer  u>  any  other  elements  in  die  inputs.  The  fact  that  the  registration  property  is 
universally  satisfied  insures  that  it  makes  sense  u>  talk  about  the  interaction  between  the  nth  values  in  all  of 
the  sequences  in  a  loop  expression  as  typical  because  they  arc  computed  from  each  other. 

Ihc  only  sequence  functions  where  there  is  any  logical  difficulty  in  satisfying  the  registration  property  are 
filters.  It  would  be  perfectly  reasonable  to  say  that  a  filter  takes  in  a  sequence  and  produces  a  shorter 
sequence  containing  only  selected  elements  of  the  input  sequence.  From  the  point  of  view  of  the  cxprcssional 
mcUiphor  there  is  nothing  wrong  with  this  definition,  and  there  would  be  no  difficulty  in  understanding  a 
program  like  SUM- POS I T I VE - EXPRESSIONAL  based  on  this  definition. 

However,  if  filters  produced  shortened  sequences,  they  would  not  satisfy  die  registration  property.  In 
order  to  satisfy  this  property,  a  filter  Is  defined  as  producing  a  sequence  which  has  exactly  the  same  number  of 
slots  as  Us  input  with  the  selectivity  of  the  filter  encoded  in  the  fact  that  some  of  the  output  slots  arc  empty. 
The  selected  values  remain  in  the  same  slots  as  in  the  input  sequence.  In  order  to  make  this  work,  loop 
expressions  arc  defined  as  simply  not  operating  on  empty  slots.  ’Ibis  can  be  seen  in  the  definitions  of  the 
various  sequence  functions  and  meta  sequence  functions  presented  above.  ITie  following  general  statement 
can  be  made:  A  given  loop  subexpression  is  only  exei'uled  on  those  cycles  of  the  loop  when  values  are  available 
for  all  of  the  sequences  it  refers  to. 

In  order  to  appreciate  the  full  impact  of  the  definition  of  how  filters  operate,  one  must  consider  loop 
expressions  involving  several  sequences.  Consider  the  program  LIST- EVEN-SQUARES.  It  takes  in  a  list  and 
returns  a  list  'Hie  output  list  contains  an  entry  corresponding  to  each  even  number  in  the  inpuL  Each  entry 
consists  of  the  number  followed  by  its  square.  For  example  when  passed  the  argument  (1  2  3  4 )  the 
program  will  produce  the  output  ( ( 2  4)  (4  16 ) ) . 

(dafun  Ust-even-squares  (list) 

(lets  ((Integers  (El  1 st  list))) 

(HI  1st  (list  (filters  f'evenp  Integers) 

(•  Integers  Integers))))) 

In  the  program,  the  function  LIST  is  implicitly  MAPSed  over  two  sequences.  The  fira  is  generated  by 
enumerating  the  elements  in  the  input  and  selecting  the  even  elements  (e.g..  [_  2  «  <])•  Ibe  second  is 
generated  by  squaring  all  of  the  elements  in  the  list  (e.g.,  [l  4  9  16]).  The  registration  between  the  two 
sequences  is  maintained  by  the  fact  that  the  missing  elements  in  the  filtered  sequence  are  represented  as 
empty  slots.  The  function  LIST  is  only  executed  when  values  are  available  in  both  sequences  i.c.,  only  for 
even  elements  of  the  list.  Ibc  output  of  the  implicit  MAPS  is  a  sequence  which  has  values  in  it  corresponding 
to  the  times  when  LIST  was  executed  (e.g.,  (_(24)_(4  16)1).  When  RUST  reduces  this  sequence  to  a  list  it 
ignores  these  empty  slots. 

The  registration  property  makes  loop  expressions  easy  to  understand  and  compile:  however,  it  is 
significantly  restrictive.  The  key  limitation  is  that  there  are  a  number  of  quite  logical  operations  on  sequences 
which  cannot  be  supported.  In  particular,  operations  which  disturb  the  ordering  of  the  slots  are  prohibited. 
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For  example,  merging  sequences,  concatenating  sequences,  changing  the  order  in  a  sequence  (c.g.,  reversing 
it),  etc.  Such  complex  operations  arc  not  supported  because  the  overhead  associated  with  supporting  them  is 
not  warranted  by  the  rather  infrequent  need  for  them.  When  they  are  needed,  other  loop  representations 
should  be  used. 

Termination 

As  discussed  above,  termination  presents  problems  from  the  point  of  view  of  the  cxprcssional  metaphor. 
A  loop  expression  is  defined  to  terminate  as  soon  its  the  shortest  sequence  in  the  expression  is  exhausted.  'Ibis 
definition  really  only  makes  sense  from  the  point  of  view  of  the  element  at  a  time  metaphor.  From  the  point 
of  view  of  the  latter  metaphor,  a  loop  expression  is  executed  by  computing  all  of  the  sequences  in  it  in  parallel 
an  element  at  a  time.  The  loop  stops  as  soon  as  any  sequence  runs  out  of  elements. 

An  interesting  aspect  of  termination  is  die  way  it  interacts  with  the  registration  property.  Suppose,  for 
example,  that  filters  produced  sequences  of  reduced  length  as  their  output.  In  this  situation,  a  filter  might 
well  produce  as  the  third  and  final  element  of  its  output  the  sixth  element  of  its  input  I  he  definition  of 
termination  above  would  require  that  the  loop  stop  after  three  cycles.  However,  this  is  a  contradiction 
because  tire  loop  must  run  for  at  least  six  cycles  in  order  to  generate  the  sixth  input  to  the  filter  so  that  the 
filler  can  produce  its  third  output. 

Note  that  all  of  (he  sequence  functions  and  meta  sequence  functions  (with  the  exception  of  TRUNCATES) 
arc  carefully  restricted  so  that  the  lengths  of  their  output  sequences  (if  any)  arc  exactly  the  same  as  the 
minimum  of  the  lengths  of  their  input  sequences  (if  any),  litis  is  done  so  that  truncators  wilt  be  the  only 
sequence  functions  which  ever  trigger  termination.  As  a  result,  reasoning  about  termination  can  focus  on  the 
truncators  in  a  loop. 

At-start,  Al-end,&  At- unwind 

In  addition  to  element  at  a  time  computation  on  sequences,  many  sequence  functions  specify  initializing 
computation  which  is  performed  before  the  loop  as  a  whole  gets  underway  and/or  epilog  computation  which 
occurs  after  the  main  body  of  the  loop  has  terminated.  Three  additional  meta  sequence  functions  are 
provided  which  make  it  possible  for  users  to  specify  computation  to  be  performed  at  these  times. 

The  meta  sequence  function  (AT -END  Junction  arg ...)  specifics  that  Junction  should  be  executed  after  the 
loop  terminates.  All  of  the  args  and  any  free  variables  referenced  by  Junction  must  hold  unitary  values.  The 
value  produced  by  applying  Junction  to  args  is  returned  as  the  unitary  value  of  the  form  as  a  whole.  As  an 
example,  consider  the  following  definition  of  RUST.  A  reducer  is  used  to  CONS  together  the  items  in  the 
input  sequence.  After  the  reduction  is  completed,  the  function  NREVERSE  is  used  to  reverse  the  order  of  the 
list  so  that  the  correct  result  is  returned. 

(defunS  Rllst  (gsequence  Items) 

(at-end  #'nr«varsa  (reduces  f'xcons  nil  Items))) 

The  meta  sequence  function  ( AT  -UNWIND  jit  net  ion  arg... )  is  just  like  AT-END  except  for  two  things.  First, 
it  produces  no  result  at  ail.  It  is  executed  for  side-effect  only.  Second,  it  will  be  executed  no  matter  l>ow  the 
loop  is  terminated.  Ihc  next  section  describes  several  situations  in  which  unusual  loop  exits  prevent  at-end 
compulation  from  being  performed.  As  an  example  of  AT- UNWIND,  consider  the  following  definition  of 
RFILE.  A  unitary  auxiliary  variable  is  used  to  hold  the  file  object  which  is  opened  to  receive  the  output  This 
file  is  closed  when  the  output  is  completed.  AT-UNWIND  is  used  because  it  is  important  to  close  the  file  no 
matter  how  the  loop  is  exited.  Note  that  A1  -  END  is  used  to  specify  that  T  should  be  returned  as  the  result  of 
RFILE. 
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(defunS  Rflle  (name  &sequenca  Items  &aux  (unitary  (file  (open  name  ’out))) 
(at-unttlnd  #'(1ambda  ()  (close  file))) 

(maps  0'( lambda  (Item) 

(let  (prinlevel  pr -in length) 

(print  Item  file))) 

Items) 

(at-end  #'(lambda  ()  T))) 

The  mctu  sequence  function  ( AT  -ST  ART  function  arg ... )  is  exactly  the  same  as  AT-END  except  that 
function  is  executed  before  the  loop  begins  rather  than  after  it  terminates.  The  initialization  of  the  auxiliary 
variable  FILE  in  RFILE  is  an  implicit  example  of  AT-START  computation.  It  could  be  made  explicit  as 
follows: 

(defunS  Rflle-expllclt-at-start  (name  (sequence  Items  (aux  (unitary  file) 
(at-start  #'( lambda  ,( )  (setq  file  (open  name  'out)))) 

(at-unwlnd  #'(lambda  ()  (close  file))) 

(maps  #'( lambda  (Item) 

(let  (prinlevel  prlnlength) 

(print  Item  file))) 

Items) 

(at-end  #'( lambda  ()  T))) 

The  examples  above  illustrate  the  most  important  use  of  AT-START.  AT-END.  and  AT -UNWIND.  They  arc 
used  to  include  initializing  and  epilog  computation  as  part  of  an  individual  sequence  function.  This  extends 
the  range  of  loop  compulation  fragments  which  can  be  expressed  as  sequence  functions.  For  example,  RFILE 
would  be  conceptually  much  less  useful  if  it  did  not  encapsulate  the  actions  of  opening  and  closing  the  file 
into  the  same  unit  with  printing  out  the  objects. 

The  program  PRINT-LIST-SUM  illustrates  the  use  of  AT-START  and  AT-END  inside  of  a  LETS.  The  first 
line  in  the  body  of  the  LETS  is  executed  only  once  at  the  start  of  the  loop  and  prints  a  heading.  Ihc  last  line  is 
executed  only  once  at  the  end  and  prints  the  sum.  The  middle  two  lines  arc  executed  on  every  cycle  of  the 
loop  and  print  out  the  integers  in  the  list  separated  by  spaces. 

(defun  print-list-sum  (list) 

(lets  ((x  (El  1st  list))) 

(at-start  *'(iambda  ()  (format  T  "-Xlntegers:  "))) 

(maps  #’(lambda  (z)  (format  T  "-D"  *))  x) 

(maps  #’( lambda  ()  (format  T  "  "))) 

(at-end  «'(lambds  (a)  (format  T  "-XThalr  Sum:  -D-X"  *))  (Rsum  x)))) 

The  output  produced  by  (print-1  let-sum  •  (l  2  j  4)) 

Integers:  1234 

Thai r  Sum:  10 
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In  order  to  make  it  easier  to  write  loops  like  the  one  above,  the  rule  for  implicit  MAPS  introduction  is 
extended  by  saying  that  every  top  level  unitary  expression  in  a  LETS  or  DEFUNS  body  is  MAPSed  if  possible. 
The  only  time  it  is  not  possible  is  if  (like  the  last  line  of  the  LETS  below)  the  expression  refers  to  a  value  which 
is  produced  by  a  reducer  and  therefore  not  available  until  after  the  loop  is  completed.  In  this  ease.  AT-END  is 
implicitly  introduced.  Note  that  neither  AT-STARI  nor  AT -UNWIND  is  ever  implicitly  introduced. 

(defun  print-1 Ist-sum-lmpl Iclt  (list) 

(format  T  "-Xlntagara:  ") 

(lets  ((x  (El  1st  list))) 

(format  T  "~D"  x) 

(format  T  -  ") 

(format  T  "~XTha1r  Sum:  ~D~XH  (Rsum  x)))) 

Looking  at  the  program  PRINT-LIST-SUM-  IMPLICIT  above  (which  is  exactly  equivalent  to  PRINT -LIST- 
SUM)  several  points  should  be  considered.  The  computation  to  be  performed  AT-START  is  simply  placed 
outside  and  before  the  LETS.  The  second  line  in  the  LETS  is  implicitly  MAPSed  even  though  it  refers  to  no 
sequence  values  at  all.  Ihc  third  line  is  implicitly  AT-ENDed  because  it  refers  to  the  output  of  a  reducer.  It 
should  be  noted  that  it  is  almost  never  necessary  to  actually  write  an  explicit  MAPS.  AT-START,  or  AT-END. 

Done 

In  addition  to  using  the  meta  sequence  function  TRUNCATES  to  limit  the  length  of  a  sequence,  a  loop  can 
also  be  terminated  by  executing  the  special  form  DONE.  Consider  the  program  SUM-INITIAL.  It  takes  in  a  list 
and  adds  up  any  initial  group  of  numbers  (c.g..  (SUM- INITIAL  '(1  2  A  4))  returns  3).  The  program 
works  by  enumerating  the  elements  in  the  list  and  summing  them  up.  but  terminating  the  loop  as  soon  as  a 
non-number  is  encountered.  The  form  (DONE)  causes  the  immediately  enclosing  loop  to  terminate  normally 
-  any  AT-END  loop  computation  which  has  been  specified  is  performed,  and  the  return  value  which  is 
specified  by  the  last  line  is  returned  (here  the  sum).  Note  that  DONE  only  makes  sense  from  the  point  of  view 
of  the  element  at  a  time  metaphor,  it  does  not  fit  into  the  cxpressional  metaphor  at  all. 

(defun  sum-initial  (Hat) 

(lets  ((x  (El  1st  list))) 

(cond  ((not  (numberp  x))  (done))} 

(Rsum  x))) 

DONE  can  also  be  called  with  one  or  more  arguments.  In  this  ease  the  loop  is  immediately  terminated  and 
the  specified  values  arc  returned.  When  DONE  is  used  in  this- way,  it  overrides  the  outputs  specified  in  the  last 
line  of  the  LETS  and  any  AT  -END  computations  are  not  performed.  An  example  of  this  use  of  DONE  is  shown 
in  the  program  find-positive  which  returns  the  first  positive  number  in  atisL 

(defun  f Ind-posltlva  (Hat) 

(lets  ((x  (Eilat  Hat))) 

(cond  ((pluap  x)  (done  x ) ) ) ) ) 

T  he  use  of  OONE  is  also  illustrated  by  the  following  sequence  functions.  The  sequence  function  ROR 
computes  the  OR  of  a  sequence  in  ihc  obvious  way  by  successively  ORing  each  value  into  a  stale  variable.  The 
first  non*NiL  value  encountered  is  returned.  Hie  sequence  function  ROR-FAST  also  returns  the  first  non-NIL 
value  encountered;  however,  it  causes  the  loop  as  a  whole  to  terminate  as  soon  as  this  value  is  found.  Note 
the  way  the  OONE  overrides  the  NIL  which  is  returned  AT-END  if  no  non-NIL  items  are  found. 
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(dsfunS  Ron  ((sequence  Item) 

(reduces  for  nil  Item)) 

(defunS  Ror-fast  ((sequence  Item) 

(cond  (Item  (done  Item))) 

(at-end  ^’(lambda  ()  nil))) 

In  general,  ROR-FAST  is  more  efficient  than  ROR:  however,  when  you  use  it  you  must  consider  the  effect 
that  it  will  have  on  the  rest  of  the  loop  it  is  used  in.  For  example,  because  its  operation  is  peremptorily 
terminated  by  the  ROR-FAST,  the  program  PRINT-LIST-OR-BUGGY  neither  prints  out  all  of  the  elements  in 
the  list,  nor  prints  out  the  summary  line  AT-END.  In  order  to  operate  as  intended,  it  needs  to  use  ROR  instead 
of ROR-FAST. 

(defun  print-1 1st-or-buggy  (list) 

(format  T  "-XE laments:  H) 

(lets  ((x  (El  1st  list))) 

(format  T  "~A  "  x) 

(format  T  M~XThe1r  Or:  ~D~X"  (Ror-fast  x)))) 

The  output  produced  by  (prlnt-llst-or-buggy  '(NIL  A  NIL  B)) 

Elements:  NIL  A 

It  should  be  noted  that  you  can  cause  die  premature  termination  of  a  loop  in  other  ways  which  arc  outside 
the  scope  of  the  LETS  macro  package.  For  example,  you  can  wrap  the  loop  in  a  PROG  and  then  do  a  RETURN 
or  GO  from  inside  the  loop  to  outside  the  loop.  (Note  that  the  loop  expression  itself  is  implemented  by  means 
of  a  PROG.  In  the  UspMachinc  version  (but  not  the  Macl.isp  version),  this  PROG  is  named  T  in  order  to  reduce 
interference  with  user  specified  RETURNS.)  Similarly,  you  can  do  a  THROW  from  inside  the  loop  to  some  CATCH 
outside  the  loop.  An  important  aspect  of  these  kinds  of  exits  is  that  they  do  not  cause  normal  termination  of 
the  loop.  No  AT-END  loop  computation  will  be. run.  and  the  return  value  is  directly  specified  by  the  RETURN 
or  THROW. 


Restart 

It  is  occasionally  convenient  to  be  able  to  restart  a  loop  at  the  beginning.  The  function  RESTART 
reinitializes  the  immediately  enclosing  loop  and  causes  it  to  start  execution  again  from  the  beginning.  Ilic  use 
of  this  function  is  illustrated  in  by  the  program  RELAX. 

(defun  relax  (graph  function) 

(1»tS  () 

(cond  ((Ror  (funcall  function  (Egraph  graph)))  (restart))))) 

The  program  RELAX  takes  in  a  graph  and  a  function.  I  he  function  is  assumed  to  take  in  a  node  of  the 
graph  and  perform  some  computation  which  may  or  may  not  result  in  side-effects  which  alter  the  node.  If  it 
alters  the  node  then  the  function  returns  T,  otherwise  it  returns  NIL.  RELAX  repetitively  applies  the  function 
to  the  nodes  of  the  graph  until  the  graph  reaches  a  quiescent  state  where  (he  nodes  arc  no  longer  changing.  A 
typical  example  of  the  way  RELAX  could  be  used  would  be  to  propagate  some  information  through  the  graph. 

The  program  works  through  multiple  passes  over  the  graph.  It  is  assumed  that  the  sequence  function 
EGRAPH  enumerates  the  nodes  of  the  graph.  In  each  pass,  the  function  is  applied  to  all  of  the  nodes  in  the 
graph.  The  program  computes  the  OR  of  the  results  of  all  of  these  function  applications.  If  any  of  them  is  T 
then  the  loop  is  restarted  in  order  to  begin  another  pass  over  the  graph. 
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LetS 

Kuch  LETS  form  delineates  the  exact  extent  of  a  loop.  All  of  the  sequence  expressions  in  it  (including  any 
expressions  specifying  values  for  the  sequence  variables  being  bound)  are  combined  together  into  a  single 
loop  which  is  separate  from  all  other  loops.  Ihc  requirement  that  the  output  of  a  LETS  be  a  unitary  value 
results  from  the  fact  that  the  each  LETS  is  compiled  into  a  separate  loop  and  therefore  cannot  be  allowed  to 
return  a  sequence  into  the  surrounding  environment 

LETS  is  defined  to  be  a  rigid  boundary  in  order  to  better  support  the  element  at  a  time  metaphor,  lurch 
LETS  delineates  a  set  of  sequences  which  will  be  processed  in  parallel.  This  is  important  for  clarifying 
concepts  such  its  AT-START  and  AT-ENO.  In  addition,  as  will  be  discussed  in  detail  below,  it  is  even  more 
important  when  considering  sidc-cflccts  (such  as  input/output)  and  nested  loops. 

It  is  typical  for  LETS  to  be  used  for  loops  which  have  a  very  strong  element  at  a  time  flavor.  In  general, 
heavy  use  is  made  of  implicit  MAPS  and  AT- END  in  order  to  express  these  loops.  The  program  INVENTORY- 
REPORT  below  shows  a  typical  example  of  using  LETS  for  this  kind  of  loop.  TTic  program  reads  in  a  file  of 
inventory  records  and  prints  out  a  report  Fach  record  is  a  list  of  four  fields:  the  name  of  the  inventory  item, 
die  quantity  on  hand,  the  minimum  acceptable  quantity  on  hand,  and  the  unit  price.  For  each  item  the  report 
prints  out  its  name,  how  many  arc  on  hand,  and  the  valuation  of  these  items  based  on  the  specified  price.  The 
last  line  of  the  output  reports  the  total  valuation  of  all  of  the  items.  In  addition  to  the  above,  the  report  prints 
out  a  notification  in  front  of  each  item  which  is  understocked  indicating  how  many  should  be  ordered. 


Sample  Inventory  File  Contents 


("Widget"  8.  8. 

2:0.6) 

("Frob"  2.  9. 

9.88) 

("Thlngy"  312.  40. 

19.68) 

("Dingus"  0.  20. 

6.28) 

("Whatsit"  3.  7. 

6.67) 

Resulting  Printout 

Inventory  Report 

• 

Order?  Name 

On  Hand 

Valuation 

Widget 

6 

$184.00 

Order:  7  Frob 

2 

$19.36 

Thlngy 

312 

$6130.80 

Order:  20  Dingus 

0 

$0.00 

Order:  4  Whatsit 

3 

$17.01 

Total  Valuation: 

$6331.17 

liking  at  the  loop  in  the  program,  note  the  use  of  dcstructuring  and  sequential  assignment  in  die  bound 
variable  value  pairs.  In  the  first  line  of  die  LETS,  the  sequence  variable  NAME  is  bound  to  a  sequence  of  the 
first  field  of  each  Tecord,  the  variable  QUANT  ITY  is  bound  to  a  sequence  of  the  second  field  of  each  record,  etc. 
Ihc  variable  valuation  is  bound  to  a  sequence  of  products  of  PRICE  and  QUANTITY. 
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(defun  Inventory-report  () 

(rith-open-f lie  (report  "Inventory .report"  ' :out) 

(format  report  "-10T  Inventory  Report~2X") 

(format  report  "Order?  Name  On  Hand  Valuation-*") 

(lets  (((name  quantity  minimum  price)  (Efile  "Inventory. data")) 

(valuation  (•$  price  (float  quantity)))) 

(cond  ((>*  quantity  minimum)  (format  report  "-10X")) 

(T  (format  report  "Order:  -3D"  (-  minimum  quantity)))) 

(format  report  "~X~1QA~4D~2X~10<$~$~>~X"  name  quantity  valuation) 

(format  report  "-X~llXTota1  Valuation :-10<$~$->"  (Rsum$  valuation))))) 

'llie  body  of  the  LETS  prinLs  the  main  part  of  the  actual  report.  The  first  form  prints  the  ordering 
notifications.  It  compares  the  quantity  in  stock  with  the  minimum  required  and  prints  out  the  number  to  be 
ordered  if  the  quantity  is  less  than  the  minimum.  The  second  form  prints  the  main  information  about  c;ich 
inventory  item.  (Note  that  the  FORMAT  function  is  a  Lisp  function-  for  creating  formatted  output.  1  .ike  the 
Fortran  construct  it  Is  modeled  after,  it  is  inscrutable  but  convenient.)  Both  of  the  first  two  forms  in  the  body 
arc  implicitly  MAPScd.  i’hc  third  form  prints  out  the  summary  line  at  the  end  of  the  report.  It  is  only 
executed  once  at  the  end  of  the  loop  because  it  uses  the  unitary  output  of  the  reducer  RSUMS  (floating  point 
sum). 

An  important  tiling  to  note  about  INVENTORY-REPORT  is  that  although  the  process  of  actually  printing  out 
the  report  (i.c.,  opening  the  file,  printing  some  initial  lines,  printing  a  group  of  internal  lines,  printing  a  final 
line,  and  then  closing  the  file)  is  clearly  a  logically  identifiable  loop  fragment,  it  is  not  represented  as  a 
sequence  function.  The  problem  is  that,  unlike  the  simpler  actions  represented  by  RFILE.  there  arc  so  many 
ways  in  which  the  items  to  be  printed,  and  the  format  for  printing  them,  can  vary  that  there  is  very  little 
constant  structure  which  could  be  captured  in  a  sequence  function.  Basically.  Uie  only  thing  which  is 
common  between  different  instances  of  this  fragment  is  opening  and  closing  the  file  which  is  already  captured 
in  the  form  WITH-OPEN-FILE. 

A  key  aspect  of  LETS  is  that  even  though  the  operation  of  actually  printing  the  report  arc  not  represented 
as  a  sequence  function,  LETS,  makes  it  possible  for  them  to  be  conveniently  expressed.  This  is  done  in 
basically  the  same  way  that  it  would  be  done  in  an  ordinary  looping  notation  i.c.,  by  distributing  the  parts  of 
the  computation  into  places  where  they  will  be  executed  in  the  correct  situations.  It  must  be  said  that  this 
makes  this  particular  fragment  no  easier  to  understand  than  it  would  be  in  an  ordinary  looping  notation. 
However,  the  loop  as  a  whole  is  more  understandable  because  much  of  the  computation  is  represented 
concisely  in  terms  of  sequence  functions.  I'hc  ability  to  mix  computations  which  arc  not  specified  as  sequence 
functions  into  a  loop  expression  is  another  important  capability  which  is  facilitated  by  the  element  at  a  time 
metaphor.  ’I’hc  issue  of  die  kinds  of  loop  fragments  which  cannot  be  represented  will  be  discussed  more  fully 
in  the  section  on  the  domain  of  applicability  of  the  expressiona!  loop  notation. 

Side- Effects 

The  behavior  of  side-effect  producing  operations  (such  as  input/output)  in  a  loop  expression  can  only  be 
understood  from  die  point  of  view  of  the  clement  at  a  time  metaphor.  This  is  another  important  reason  why 
this  metaphor  is  made  a  prominent  part  of  the  description  of  the  notation. 

The  compilation  process  is  constrained  so  that  the  order  of  execution  in  the  loop  produced  is  rigidly  linked 
to  the  lexical  order  of  expressions  in  the  original  loop  expression.  As  a  result,  it  is  relatively  easy  to  predict  the 
consequences  of  side-cffccts  as  long  as  you  bear  in  mind  the  fact  that  processing  is  occurring  an  clement  at  a 
time  so  that  the  side-effect  operations  are  interleaved  and  that  each  one  is  executed  many  times.  Consider  the 
program  INVENTORY-REPORT  above.  I'hc  two  main  output  statements  arc  each  executed  once  on  each  cycle 
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of  the  loop.  The  final  output  statement  is  executed  only  twice  after  the  loops  terminates. 

Note  that  die  requirement  that  every  unitary  expression  in  a  LETS  he  MAPSed  if  possible  was  introduced  in 
order  to  make  side-effects  easier  to  understand.  One  might  have  said  that  an  expression  which  neither  uses 
nor  produces  sequence  values  should  not  be  MAPSed  since  its  value  cannot  change  on  different  cycles  of  the 
loop.  However,  this  would  be  missing  the  fact  that  if  it  has  a  side-effect  (such  as  output  to  a  file)  this  effect  is 
probably  desired  on  every  cycle  of  the  loop.  A  programmer  can  use  AT-START  in  order  to  specify  that 
something  should  only  be  executed  once. 

Most  side-effects  interact  with  die  cxprcssional  notation  in  straightforward  ways  and  can  easily  be 
understood  as  outlined  above.  However,  there  are  some  situations  where  diings  arc  not  so  clear. 

Side-Effects  and  Termination 

In  older  to  understand  how  side-effects  interact  with  termination,  one  has  to  be  aware  of  exaedy  when 
termination  will  occur.  For  example,  consider  the  program  PRINT-LIST  below.  I  bis  function  prints  all  of 
the  items  in  a  list  preceding  each  one  with  an  index  of  its  position  in  the  list 

(defun  print-list  (list) 

(lets  ((1  (generates  r  1+  1)) 

(x  (El  1st  list))) 

(format  T  "~%Item  ~0:"  1) 

(format  T  ”  ~A”  x) ) ) 

The  output  produced  by  (print-list  ’(A  B  C)): 

Item  1:  A 
Item  2:  B 
Item  3:  C 

There  is  one  potential  pitfall  which  the  user  must  be  aware  of.  A  loop  is  terminated  immediately  upon 
discovering  that  one  of  die  sequences  has  been  exhausted.  As  a  result  of  this,  unless  the  termination  test 
happened  to  be  the  first  thing  executed  on  that  cycle  of  the  loop,  some  diings  will  get  executed  on  that  last 
cycle,  and  others  will  not.  In  particular,  all  and  only  those  expressions  which  lexically  precede  die  termination 
will  be  executed.  For  example,  consider  die  program  PRINT-LIST-BUGGY.  (Note  that  although  no  sequence 
variables  arc  bound,  a  LETS  is  required  in  this  program  in  order  to  specify  dial  the  two  FORMATS  should  be 
executed  in  a  single  loop  instead  of  in  two  separate  loops.  The  LETS  also  specifics  dial  the  FORMATS  should  be 
MAPSed.) 

(dafun  print-list-buggy  (list) 

(lets  () 

(format  T  H~XItem  ~D:"  (generates  #’1+  1)) 

(format  T  "  -A"  (Elist  list)))) 

'Hie  output  produced  by  (prlnt-llst-buggy  '(A  B  C ) ) : 

Item  1:  A 
Item  2:  8 
Item  3:  C 
Item  4: 

This  program  docs  not  produce  die  same  output  as  PR  I  NT -LIST.  The  problem  is  that  it  docs  not  discover 
that  the  list  has  been  exhausted  until  after  the  first  FORMAT  has  been  executed  on  the  last  cycle  of  the  loop. 
Note  that  Uiis  problem  cannot  be  avoided  by  any  straightforward  change  to  the  definition  of  LETS.  You 
could  not  sav  that  nothing  in  a  cycle  will.be  executed  if  any  termination  is  triggered  because  some  of  the 
computation  ina>  be  necessary  in  order  to  compute  when  to  terminate.  On  the  other  hand,  you  could  not  say 
d  al  everything  will  be  executed  on  the  cycle  where  termination  occurs  because  typically  some  (or  all)  of  the 
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iputation  after  the  termination  test  will  be  in  error  if  die  test  is  true. 

The  programmer  is  capable  of  exercising  control  over  this  problem  because,  in  the  loop  code  which  is 
duccd.  everything  is  evaluated  in  the  order  in  which  it  appears  in  the  original  loop  expression.  As  a  result, 
;  always  possible  for  him  to  get  the  termination  tests  to  occur  at  the  places  he  wants  by  correctly  ordering 
forms  in  the  LETS.  For  example,  die  ELIST  is  merely  placed  before  die  first  FORMAT  in  PRINT-LIST.  As 
suit,  this  is  not  really  a  severe  problem;  however,  it  is  one  to  which  the  user  must  be  sensitized. 

On  a  deeper  level,  the  real  problem  with  PRINT-LIST-BUGGY  is  that  neither  it  (nor  for  dial  matter  PRINT- 
IT)  makes  the  logical  relationship  between  die  two  FORMATS  explicit,  flic  correct  tiling  to  do  is  to  group 
in  together  into  a  single  form  as  in  die  function  PRINT -LIST-BEST. 

(defun  print-1 ist-best  (list) 

(lets  () 

(format  T  "~%Item  ~D:  ~A"  (generates  #’1+  1)  (El  1st.  list)))) 


Side* Effects  Between  Sequence  Functions 

As  mentioned  above,  the  exprcssional  notation  attempts  to  maintain  the  property  of  decomposability  of 
p  expressions  whenever  possible.  An  important  feature  of  this  is  diat  any  internal  state  variables  of  a 
uence  function  arc  hidden  from  view  and  cannot  be  modified  by  SETQs.  or  the  like,  in  a  loop  expression, 
fortunately,  side-effect  producing  functions  such  as  RPLACD  are  capable  of  modifying  die  values  of  state 
iablcs  without  having  ui  actually  refer  to  the  variables  themselves.  If  such  side-effect  functions  arc  being 
d.  then  the  programmer  must  take  care  that  tliis  kind  of  problem  docs  not  arise. 

IT>c  problem  is  illustrated  by  die  program  DASH-LIST-BUGGY.  The  purpose  of  this  program  is  to  take  in  a 
(c.g.,  (A  B  C))  and  put  a  dash  after  each  entry  in  it  (c.g.,  to  produce  (A  -  B  -  C  -)).  It  attempts  to  do 
by  side-effect  as  follows.  It  enumerates  each  of  the  sublists  in  the  original  list  (c.g..  |(A  B  C)(BC)(C)J) 
splices  in  a  dash  after  the  first  element  of  each  sublist  (e.g„  producing  [(A  -  B  C)  (B  -  C)  (C  -)]). 

(defun  dash-list-buggy  (list) 

(TetS  ((sublist  (Esubllsts  list))) 

(rplacd  sublist  (cons  *-  (edr  sut,1st)))) 
list) 

Particularly  from  the  point  of  view  of  the  exprcssional  metaphor,  the  above  algorithm  sounds  very 
isiblc;  however,  it  doesn't  work.  What  actually  happens  is  that  the  program  goes  into  an  infinite  loop 
ring  in  dashes  after  the  first  item  in  the  input  list.  For  example,  if  the  loop  starts  with  die  list  (ABC) 
i  the  first  sublist  is  (A  B  C).  The  RPLACD  alters  this  sublist  to  (A  -  B  C )  and  therefore  die  list  itself  to 
-  B  C).  So  far  this  is  all  as  intended.  Unfortunately,  an  internal  variable  in  ESUBLISTS  has  a  pointer 
the  list  in  order  to  keep  track  of  what  sublist  to  enumerate.  The  list  is  altered  before  the  second  sublist  is 
ally  enumerated  and  as  a  result  ( -  B  C )  gets  enumerated  us  the  second  sublist  instead  of  ( B  C ) . 
t  is  possible  to  construct  a  loop  expression  for  this  algorithm  which  will  work  more  or  less  as  intended, 
example,  the  program  DASH-LIST1  combines  everything  into  one  enumerator  which  enumerates  die  next 
ist  before  the  RPLACD  operation.  Alternatively,  DASH- LI  ST  z  uses  a  modified  enumerator  which  makes 
vanccs  for  die  actions  of  the  RPLACD. 
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(defun  dash-11$tl  (list) 

(lets  () 

(enumerates  0'nul  1 

#’{lambda  (1)  (progl  (edr  1)  (rplacd  1  (cons  *-  (edr  1))))) 
list)) 

list) 

(defun  dash-list2  (list) 

(lets  ((sublist  (enumerates  #'null  #'cddr  list))) 

(rplacd  sublist  (cons  (edr  sublist)))) 
list) 

l  lowcvcr,  due  to  the  antagonistic  interaction  between  the  RPLACD  and  the  enumerator,  there  is  no  aesthetic 
way  to  express  the  slated  algorithm  using  die  expressional  notation.  As  will  he  discussed  in  more  detail  below, 
this  is  one  of  die  kinds  of  algorithms  for  which  the  expressional  notation  is  not  intended  to  be  used. 

Conversions  and  Coercions 

Two  sequence  functions  arc  available  for  converting  between  unitary  values  and  sequences:  GSEQUENCE 
which  converts  an  object  into  an  infinite  sequence  of  that  object,  and  RLAST  which  converts  a  sequence  into  a 
unitary  object  by  taking  its  last  element.  It  should  be  noted  that  the  meta  sequence  function  MAPS  is  like 
GSEQUENCE  in  many  ways.  If  passed  a  unitary  object  it  will  also  create  an  infinite  sequence  of  that  object. 
However,  if  you  nest  a  unitary  expression  in  MAPS  it  will  be  executed  many  limes,  while  if  you  .apply 
GSEQUENCE  to  die  expression  it  will  be  evaluated  only  once.  For  example.  VECTOR-NCONS  initializes  a  vector 
by  filling  all  of  its  slots  with  the  same  CONS  cell.  In  contrast,  VECTOR-NCONSES  fills  each  slot  with  a  different 
CONS  cell. 

(defun  vector-neons  (vector) 

(Rvector  vector  (Gsequence  (neons  nil)))) 

(defun  vector-nconses  (vector) 

(Rvector  vector  (maps  ^’(lambda  ()  (neons  nil))))) 

In  order  to  make  things  more  convenient  for  the  user,  automatic  type  coercions  arc  applied  between 
sequences  and  unitary  values.  I’hc  most  important  coercion  has  already  been  discussed.  Whenever  a  unitary 
expression  is  placed  where  a  sequence  value  is  required.  MAPS  is  automatically  introduced  in  order  to  convert 
it  into  a  sequence  expression.  Note  that  GSEQUENCE  is  never  automatically  introduced  and  therefore  VECTOR- 
NCONSES- IMPLICIT  is  equivalent  to  VECTOR-NCONSES.  and  not  to  VECTOR-NCONS. 

(defun  vector-nconses-lmpl Iclt  (vector) 

(Rvector  vector  (neons  nil))) 

The  places  where  sequence  values  arc  required  arc  the  sequence  arguments  to  sequence  functions  and  the 
:xprcssions  to  be  bound  to  values  in  a  LETS.  These  coercions  arc  illustrated  by  the  following  pairs  of 
tquivalent  loop  expressions. 

(Rl  1st  t) 

same  as:  ( R 1 1st  (maps  #’(lambda  ()  1))) 

fletS  ((x  1)) 

...) 

same  as:  (lets  ((x  (mapS  #’(lambda  ()  1)))) 

...) 

In  the  reverse  direction,  whenever  a  sequence  expression  is  placed  where  a  unitary  value  is  required, 
[LAST  is  automatically  introduced  u»  convert  it  into  a  unitary  value  producing  expression.  I’hc  places  where 
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unitary  values  arc  required  arc  the  last  expression  in  the  body  of  a  LETS  and  the  value  of  loop  expressions 
which  appear  in  isolation  in  ordinary  l.isp  code.  Examples  arc  shown  below. 

(lets  ((x  (Elist  list))) 

(mapS  0'prlnt  x)) 
same  as:  (lets  ((x  (Elist  list))) 

(Rlast  (mapS  0'print  x))) 

(maps  #'pr1nt  (Elist  x)) 
same  as:  (Rlast  (maps  if'print  (Elist  x))) 

Some  of  the  other  features  of  the  cxprcssional  notation  could  also  be  looked  at  as  coercions.  Tor  example, 
the  automatic  introduction  of  MAPS  and  AT-END  around  lines  ofa  LETS.  Taken  together,  these  coercions  have 
no  semantic  import  --  they  do  not  make  it  possible  to  express  anything  which  could  not  be  expressed  without 
them.  However,  they  do  make  it  significantly  more  convenient  to  specify  many  kinds  of  loops. 


Nested  Loops 

Like  any  looping  notation,  the  cxprcssional  notation  can  be  used  U)  express  nested  loops.  Consider  the 
program  SUM-LISTS-IN-LISTl.  It  takes  in  a  list  of  lists  of  integers  (c.g.,  ((1  2)  (3  4) ))  and  returns  a  list 
of  the  sums  of  these  lists  (c.g.,  (3  7)).  The  outer  loop  enumerates  the  lists  of  numbers  in  die  list  supplied  as 
the  input  to  die  function  as  a  whole.  The  inner  loop  adds  up  the  numbers  in  these  sublists.  'Ihc  outer  loop 
then  CONScs  these  numbers  up  into  a  list  to  be  returned. 

(defun  sum-1 isfs-ln-1 Istl  ( 1 1st-of-l Ists) 

(lets  ( (entry  (Elist  1 ist-of-1 Ists) ) ) 

(setq  entry  (mapS  ^’(lambda  (l)-(Rsum  (Elist  1)))  entry)) 

(Rllst  entry))) 

In  the  program.  MAPS  is  used  to  apply  die  inner  loop  to  each  list  of  numbers  in  turn.  The  LAMBDA  used 
with  the  MAPS  delineates  the  boundary  of  the  inner  loop.  This  could  also  be  done  by  wrapping  a  LETS  around 
the  inner  loop  (which  would  then  be  implicitly  MAPSed)as  in  SUM-LISTS-IN-LIST2. 

.tjfun  sum-lists-1n-11st2  (1 Ist-of-llsts) 

(lets  ((entry  (Elist  1 lst-of-1 Ists) )) 

(setq  entry  (lets  ()  (Rsum  (Elist  entry)))) 

(Rllst  entry))) 

Though  relatively  clear,  both  of  the  above  programs  arc  somewhat  cumbersome  in  appearance.  If  the 
(RSUM  (ELIST  . . . ))  were  in  isolation,  there  would  be  no  need  to  wrap  it  in  cithcra  MAPS  or  LETS.  The 
same  is  true  here.  The  algorithm  can  be  more  conveniently  expressed  as  shown  in  SUM-LISTS-IN-LIST3. 

(dsfun  sum-1 lsts-ln-1 1st3  (1 ist-of-llsts) 

(lets  ((entry  (Elist  11st-of-11sts))) 

(setq  entry  (Rsum  (Elist  entry))) 

(Rllst  entry))) 

Note  that  from  the  point  of  view  of  the  element  at  a  time  metaphor,  the  body  of  the  LETS  is  describing 
what  happens  to  a  typical  value  of  ENTRY.  This  typical  value  is  unitary  and  therefore  it  makes  perfect  sense  to 
say  that  (RSUM  (ELIST  . .. ))  is  applied  to  it.  However,  as  written,  the  limp  expression  contains  type 
conflicts.  The  sequence  ENTRY  is  supplied  where  ELIST  expects  a  unitary  input,  and  the  unitary  output  of 
RSUM  is  assigned  to  the  sequence  ENTRY.  In  order  to  deal  with  this,  an  automatic  conversion  is  applied  which 
parses  each  loop  expression  looking  for  matched  pairs  of  type  conflicts  like  dicse.  'lltcsc  conflicts  arc  then 
resolved  by  separating  out  a  nested  loop  which  is  MAPSed  over  the  input  sequence. 

Unfortunately,  there  arc  several  major  problems  involved  with  the  automatic  introduction  of  nested  loops 
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as  described  above.  live  first  is  that  although  the  third  version  of  the  program  above  is  arguably  more 
readable  than  die  first  two  versions,  the  process  of  representing  a  loop  more  and  more  compactly  can  easily  by 
carried  to  excess.  For  example,  the  next  two  versions  of  die  program  are  more  compact  and  specify  exactly 
the  same  computation.  However,  it  is  questionable  whether  dicy  are  easier  to  understand.  Going  beyond 
Um,  more  complex  programs  (such  as  triplely  nested  loops)  become  virtually  incomprehensible  if  rendered  in 
such  a  dense  expiessionai  style. 

(defun  sum-1 1sts-in-l 1st4  (llst-of-llsts) 

(lets  ((entry  (lilist  1  ist-of-1 1  s ts ) ) ) 

(Rlist  (Rsum  (El  1st  entry))))) 

(defun  sum-1 Ists- in-1 ist5  ( 1 ist-of-1 1 s ts ) 

(Rlist  (Rsum  (El  1st  (El  1st  1 ist-of-1 Ists) ) ) )) 

Another  problem  with  die  automatic  creation  of  nested  loops  is  that  although  die  required  parsing  is  trivial 
in  simple  cases  like  the  above,  it  is  unfortunately  quite  complex  in  the  general  ease.  One  reason  for  this  is  that 
there  is  considerable  interaction  with  the  type  coercion  processes  described  above.  Another  stems  from  the 
fact  diat  parsing  also  has  to  deal  with  die  related  phenomenon  illustrated  in  die  program  SUM-COPY-OF-LIST 
below.  This  program  copies  a  list  of  integers  and  then  computes  the  sum  of  the  integers.  Note  diat  there  are 
no  type  conflicts  in  this  program,  and  that  the  initial  copying  of  the  list  is  not  a  nested  loop.  It  is  executed  in 
its  entirety  before  die  summation  loop  begins.  Nevertheless,  the  copying  loop  has  to  be  located  and  separated 
from  die  rest  of  the  loop  since  it  is  computed  separately.  'Ihe  need  to  do  diis  further  complicates  the  parsing 
process.  AH  in  all  the  parsing/coercion  process  ends  up  being  by  far  die  most  complex  part  of  the 
compilation  process. 

(defun  sum-copy-of-1 1st  (list) 

(Rsum  (El  1st  (Rlist  (El  1st  list))))) 

A  much  bigger  problem  with  the  automatic  introduction  of  nested  loops  is  that  the  complex  interactions 
with  other  coercions  arc  not  just  hidden  inside  the  compilation  process  --  they  can  lead  to  considerable 
confusion  regarding  seemingly  simple  loop  expressions.  For  example,  consider  the  program  ZERO-MATRIX. 
As  rendered  below  it  has  a  simple  explicit  nested  loop  which  sets  all  of  the  elements  of  an  array  to  zero.  If  one 
tries  to  express  this  more  compactly,  Uiings  rapidly  become  complicated. 

(defun  zero-matrix  (A) 

(lets  ((1  (Erantje  0  (1-  (array-dlmenslon-n  1  A))))) 

(lets  ((j  (Erange  0  (1-  (array-dlmenslon-n  Z  A))))) 

(aset  0  A  1  j)))) 

The  variable  J  is  only  used  once  in  the  inner  loop,  so  the  program  ZERO-MATRIX-NO-J  is  equivalent  to 
ZERO-matrix.  However,  if  you  oinit  the  inner  LETS  as  in  ZERO-MATRIX-BUGGY  you  no  longer  have  an 
equivalent  program.  This  program  is  is  not  equivalent  to  ZERO-MATRIX,  but  rather  to  ZERO-DIAGONAL. 
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(defun  zero-matrix-no- j  (A) 

(lets  ((1  (Erange  0  (1-  (array-dimenslon-n  t  A))))) 

(lets  () 

(aset  0  A  1  (Erange  0  (1-  (array-dimenslon-n  2  A))))))) 

(defun  zero-matrlx-buggy  (A) 

(letS  ((1  (Erange  0  (1-  (array-dimenslon-n  t  A))))) 

(aset  0  A  i  (Erange  0  (1-  (array-dimenslon-n  2  A)))))) 

(defun  zero-diagonal  (A) 

(lets  () 

(aset  0  A  (Erange  0  (1-  (array-dimenslon-n  1  A))) 

(Erange  0  (1-  (array-dimenslon-n  2  A)))))) 

The  problem  is  that  there  is  a  considerable  amount  of  coercion  going  on  in  ZERO-MATRIX-NO-J  which  is 
no  longer  forced  in  ZERO-MATRIX-BUGGY.  In  particular:  the  ASET  is  MAPScd  over  die  sequence  created  by  the 
inner  ERANGE:  in  conjunction  with  this  MAPS,  each  individual  value  of  I  is  in  effect  converted  into  a  sequence 
of  identical  values:  and  RLAST  is  used  to  convert  the  sequence  of  values  created  by  the  MAPScd  ASET  into  a 
unitary  return  value.  In  ZERO-MATRIX-BUGGY  everything  can  be  interpreted  much  more  simply  by  assuming 
that  the  two  ERANGEs  arc  being  executed  in  parallel.  It  is  a  general  feature  of  LETS  that  it  only  creates  nested 
loops  when  it  is  absolutely  necessary  --  it  always  tries  to  combine  everything  in  its  body  into  a  single  loop. 

Given  the  difficulties  it  causes,  the  obvious  question  is  why  support  the  automatic  introduction  of  nested 
loops?  The  problem  is  that  the  simple  cases  of  implicitly  nested  loops  arc  so  logically  compelling  that  they 
cannot  be  ignored.  It  seems  to  be  an  obvious  benefit  to  be  able  to  use  a  simple  loop  expression  (such  as 
(RSUM  (ELIST  X)))  in  isolation  in  a  program.  Add  to  this  the  fact  that  the  clement  at  a  time  metaphor 
suggests  thinking  about  the  interior  of  a  loop  as  a  specification  for  what  happens  to  typical  (unitary)  values  of 
the  sequences  in  it,  and  programs  like  SUM-LISTS- IN -LIST3  seem  too  reasonable  to  prohibit 

A  Large  Example 

To  conclude  the  description  of  the  features  of  die  cxprcssional  loop  notation,  this  section  presents  a  larger 
example.  The  example  is  a  data  abstraction  which  implements  sets  of  symbols  as  bit  vectors.  The  abstraction 
not  only  makes  available  some  ordinary  functions  for  operating  on  these  sets,  but  some  sequence  functions  as 
well. 

Sets  are  represented  as  bits  packed  into  a  single  integer.  The  size  of  the  sets  is  limited  by  the  number  of 
bits  in  an  integer  (e.g..  24  bits  on  the  LispMachinc).  The  global  variable  •BSET-DOMAIN*  stores  the 
correspondence  between  potential  set  elements  and  bit  positions.  'Ibis  mapping  is  represented  by  a  vector  of 
CONScs.  The  index  of  a  CONS  in  the  vector  indicates  the  bit  position  which  is  being  described.  The  CAR  of  the 
CONS  holds  the  symbol  which  corresponds  to  die  bit  position.  The  CDR  of  the  CONS  holds  the  representation 
for  a  set  which  has  only  that  one  symbol  in  it  (i.c.,  an  integer  with  only  the  one  corresponding  bit  on).  The 
variable  •BSET-DOMAIN*  is  initialized  to  a  vector  of  conscs  of  NIL  and  the  appropriate  single  clement  sets. 
Note  that  the  unit  sets  arc  created  by  a  special  generator  which  starts  with  an  integer  with  a  1  in  bit  position  0 
and  then  rotates  this  bit  around  from  position  to  position. 

(defvar  *bset-doma1n* 

(Rvector  (array  nil  T  24)  (cons  nil  (generates  iT(1ambda  (x)  (rot  x  1))  I))) 
"The  bset  domain  element  mapping.”) 

The  global  variable  *BSET- INDEX*  keeps  track  of  the  largest  bit  position  used  so  far.  The  number  -1  is 
used  to  represent  die  fact  that  no  bit  positions  have  been  used  yet. 
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(defvar  •bset-lndex*  -1  "The  largest  bit  position  used  so  far.") 

The  function  8SET -RESET  is  used  to  reinitialize  these  variables.  It  MAPScs  over  the  vector  in  *BSET- 
domain*  setting  the  CAR  of  each  CONS  cell  to  NIL.  and  sets  •BSET-INOEX*  to  -1. 

(defun  bset-reset  () 

(lets  ((Item  (Evector  •bset-domaln*))) 

(setf  (car  Item)  nil)) 

(setq  *bset-1ndex*  -1)) 

The  function  BSET-UNITSET  takes  in  a  symbol  and  returns  the  unit  set  corresponding  to  it  It  issues  an 
error  if  the  symbol  is  not  representable  as  a  unit  set  (i.c..  if  it  is  not  in  the  vector  *BSET -DOMAIN*).  It  uses 
ROR-FAST  (which  as  described  above,  computes  the  OR  of  the  items  in  a  sequence,  stopping  as  soon  as  a 
non-NIL  Hem  is  encountered)  in  order  to  look  for  the  symbol  in  •BSET-DOMAIN*  returning  the  corresponding 
unit  set  as  soon  as  it  is  found.  Note  that  the  CONO  in  the  ROR-FAST  is  implicitly  MAPSed. 

(defun  bset-unltset  (symbol) 

(or  (lets  ((Item  (Evector  *bsat-doma1n«  0  *bset-1ndex*))) 

(Ror-fast  (cond  ( ( eq  (car  Item)  symbol)  (edr  Item))))) 

(error  "symbol  not  In  bset  domain"  symbol))) 

'the  function  BSET -ADD-DOMAIN-ELEMENT  takes  a  symbol  and  enters  it  in  •BSET-dohain*  so  that  it  can 
be  used  in  the  bit  vector  sets,  if  the  symbol  is  not  already  in  the  domain,  and  if  there  is  an  available  bit 
position,  then  the  program  increments  *BSET-INDEX*  and  stores  the  symbol  in  the  appropriate  CONS  cell  in 
•BSET-DOMAIN*. 

(defun  bset-add-domaln-elemant  (symbol) 

(cond  ((Ror-fast  (eq  symbol  (car  (Evector  »bset-doma1n*  0  •bset-lndex*))))) 

((>  *bset-1ndex*  22)  (error  "bset  domain  size  exceeded"  nil)) 

(T  ( Incf  *bset-1ndex«) 

(setf  (car  (aref  •bset-domaln*  *bset-1ndex* ))  symbol)))) 

As  examples  of  the  kind  of  ordinary  functions  which  would  be  implemented  as  part  of  the  data  abstraction 
consider  the  following  four.  The  first  three  are  examples  of  the  operations  for  which  tile  bit  vector 
implementation  is  particularly  efficient.  Intersection,  union,  and  the  test  for  equality  between  two  sets  can  ah 
be  implemented  as  single  operations  independent  of  how  many  symbols  are  in  the  sets  operated  on. 

(defun  bset-lntersect  (bsetl  b$et2) 

(logand  bsetl  bset2)) 

(dafun  bset-unlon  (bsetl  bsat2) 

(loglor  bsetl  bset2)) 

(defun  bset-equal  (bsetl  bset2) 

(*  bsetl  bset2)) 

(defun  bsat-mem  (symbol  bset) 

(not  (zerop  (bset-lntersect  (bset-unltset  symbol)  bset)))) 

The  next  four  definitions  arc  examples  of  die  kind  of  sequence  functions  which  would  be  provided  as  part 
of  the  data  abstraction.  The  first  two  implement  reducers  which  can  be  used  to  take  the  intersection  and 
union  ol  sequences  of  bit  vector  sets.  The  third  (EBSET)  takes  in  a  bit  vector  set  and  creates  a  sequence  of  the 
symbols  in  that  set  Ihe  Iasi  (RBSET)  performs  the  inverse  operation,  taking  in  a  sequence  of  symbols  and 
creating  a  set  by  taking  the  union  of  the  corresponding  unit  sets. 
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(defunS  Rbset-lntersect  ((sequence  bset) 

(reduces  #'(lambda  (*)  (bset-lntersect  x  bset))  -1)) 

(defunS  Rbset-unlon  ((sequence  bset) 

(reduces  #'(1ambda  (x)  (bset-unlon  x  bset))  0)) 

(defunS  Ebset  (bset) 

(car  (filters  #'(1ambda  (x)  (not  (zerop  (bset-lntersect  (edr  x)  bset)))) 
(Evector  •bsat~doma1n*  0  *bset-lndex*)))) 

(defunS  Rbset  ((sequence  symbol) 

(Rbset-unlon  (bset-unltset  symbol))) 

llic  example  above  is  a  particularly  good  one  in  that  it  shows  the  cxprcssional  notation  being  used  to 
represent  a  variety  of  loops  which  arc  small  and  simple.  This  is  the  application  for  which  the  notation  has 
been  specifically  designed. 
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Hi  -  Evaluating  the  Exprcssional  Notation 

One  way  to  summarize  the  exprcssional  loop  notation  is  as  a  collection  of  basic  ideas.  First,  there  arc  three 
themes  which  underlie  the  notation. 

The  Exprcssional  Metaphor-' Ihc  idea  that  loops  can  be  expressed  as  compositions  of  fragments  of 
looping  behavior  is  the  fundamental  motivation  behind  the  notation. 

The  Element  at  a  Time  Metaphor-  The  additional  metaphor  that  a  loop  can  be  conveniently  specified 
as  a  set  of  operations  on  typical  elements  also  underlies  the  notation  as  a  whole. 

Efficient  Compilation  -  From  the  beginning,  it  was  decided  that  it  hud  to  be  possible  to  compile  the 
nutation  into  efficient  looping  code.  This  effected  many  of  the  design  decisions. 

There  arc  six  basic  features  of  the  notation  which  together  support  these  themes. 

Science  Functions -'These  embody  the  fundamental  notion  of  a  fragment  of  looping  behavior.  'Ihc 
fact  that  they  look  and  can  be  reasoned  about  essentially  just  like  ordinary  functions  supports  the 
exprcssional  metaphor.  Restrictions  on  (he  kinds  of  sequence  functions  allowed  (c.g.,  the 
requirement  for  registration  between  elements  of  their  inputs  and  outputs)  support  die  element  at  a 
lime  metaphor  and  efficient  compilation. 

Sequences  -  These  are  the  mode  of  communication  between  sequence  functions.  ITic  fact  that  they 
look  like  and  can  be  reasoned  about  much  of  the  time  just  like  ordinary  aggregate  data  objects 
supports  the  exprcssional  metaphor.  ‘Ihc  fact  that  they  arc  defined  to  be  one  dimensional  series  of 
slots  containing  unitary  values  where  each  slot  corresponds  to  one  cycle  of  the  lixip  which  will 
eventually  be  produced  is  the  fundamental  underpinning  for  the  element  at  a  time  metaphor  and  is 
esscmi.il  for  efficient  compilation. 

User  Definition  of  Sequence  Functions  •  The  fact  that  the  user  can  define  his  own  sequence  functions  in 
analogy  with  the  definition  of  ordinary  functions  greatly  extends  the  utility  of  the  notation. 

Meta  Sequence  Functions-  ilicsc  make  it  possible  to  specify  new  kinds  of  operations  on  sequences. 

On  the  one  hand,  they  provide  a  very  convenient  mechanism  for  this  specification.  One  the  other 
hand,  diey  embody  die  restrictions  which  arc  necessary  in  order  to  insure  that  it  will  be  possible  to 
efficiently  compile  the  specified  operations.  In  this  context  it  is  important  diat  the  notation  docs  not 
provide  any  more  general  method  for  specifying  sequence  computations. 

Loop  Expression  Blocks- Calls  on  LETS  serve  two  basic  purposes:  delineating  groups  of  loop 
expressions  which  arc  to  be  combined  into  a  single  loop,  and  supporting  the  notion  of  variables 
which  have  sequences  as  their  values.  The  body  of  such  a  block  is  die  place  where  the  element  at  a 
lime  metaphor  is  most  prominent. 

Coercions -The  existence  of  coercions  such  as  the  automatic  introduction  of  MAPS  is  an  important 
underpinning  for  the  element  at  a  time  metaphor.  Other  coercions  such  as  the  detection  of  nested 
loops  and  automatic  conversions  between  sequences  and  unitary  values  exist  merely  as  a 
convenience  for  the  user.  Note  that  in  order  to  make  the  above  coercions  practical,  variables 
containing  sequences  have  to  be  readily  identifiable  as  such. 

In  order  to  investigate  the  efficacy  of  the  exprcssional  loop  notation  as  a  whole,  one  must  look  at  it  from 
several  points  of  view.  In  particular,  one  must  evaluate  the  kind  of  loops  it  can  be  applied  to,  how  efficiently 
it  can  be  executed,  and  how  easily  it  could  be  applied  to  other  languages  besides  Lisp. 
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Domain  of  Applicability 

'Che  expressions)  loop  notation  is  oriented  towards  the  kinds  of  straightforward  loops  which  arc  most 
common.  In  order  to  make  it  easier  to  express  these  loops,  it  deliberately  sacrifices  more  general  applicability. 
As  a  result,  there  are  a  number  of  situations  where  the  cxprcssinnal  notation  is  not  appropriate. 

'Ihc  basic  approach  of  the  notation  is  to  express  a  loop  as  a  composition  of  fragments  of  looping  behavior 
represented  as  sequence  functions.  'Ihcre  arc  two  main  situations  in  which  this  approach  is  ineffective:  when 
a  loop  cannot  he  separated  into  multiple  fragments,  and  when  the  notion  of  a  sequence  function  is  not 
capable  of  expressing  the  required  fragments. 

It  is  quite  possible  that  even  a  large  loop  wilt  not  be  decomposable  into  fragments.  In  order  to  break  a 
loop  down  into  two  fragments  A  and  11,  it  must  be  live  ease  that  A  and  II  arc  both  self  contained  units.  In 
particular  this  means  that  there  can  be  no  interaction  between  A  and  B  other  than  data  flow  from  A  to  B.  Note 
that  there  cannot  be  any  data  flow  from  B  to  A.  In  some  loops,  all  of  die  computation  is  linked  together  in  a 
light  net  of  data  flow,  in  that  ease  it  cannot  be  decomposed.  For  example,  consider  the  program  BINARY- 
MEN  which  tests  whether  a  given  integer  is  in  a  sorted  vector  of  integers  by  doing  a  binary  search. 

(defun  binary-mam  (Integer  vector) 

(prog  (left  mid  right  Item) 

(setq  left  0) 

(setq  right  (1-  (array-length  vector))) 

L  (cond  ((>  left  right)  (return  nil))) 

(setq  mid  (//  (+  left  right)  2)) 

(setq  item  (aref  vector  mid)) 

(cond  ((>  item  Integer)  (setq  right  (1-  mid))) 

((<  item  Integer)  (setq  left  (1+  mid))) 

(T  (return  T))) 

(90  L))) 

Ihc  program  cannot  be  decomposed  into  a  composition  of  fragments  because  each  part  affects  every  other 
pan.  The  values  of  LEFT  and  RIGHT  arc  used  to  compute  MID  which  is  used  to  compute  ITEM  which  is  used  in 
a  test  which  determines  the  next  values  of  LEFT  and  RIGHT.  Because  it  cannot  be  decomposed,  there  is  no 
way  to  write  the  program  any  more  clearly  using  the  cxprcssional  notation.  'Ihc  best  that  could  be  done 
would  be  to  write  the  program  as  one  huge  sequence  function. 

Often  sidc-cffccts  tic  together  parts  of  a  loop  which  might  appear  to  be  separable.  An  example  of  this  was 
shown  in  the  section  on  side-effects  above.  'Ihc  basic  algorithm  presented  there  for  adding  dashes  into  a  list 
cannot  be  decomposed  because  the  RPLACDs  performed  on  the  sublisls  which  are  enumerated  modifies  the 
state  of  the  enumerator.  The  only  solutions  are  either  to  represent  the  program  as  a  single  fragment  as  in 
DASH-LISTl,  or  to  write  a  program  like  DASH-LISTZ  where  the  loop  appears  to  have  been  decomposed  but 
actually  has  not.  Both  approaches  arc  unsatisfactory.  Neither  program  is  particularly  easy  to  understand,  and 
the  second  program  violates  the  basic  spirit  of  the  cxprcssional  notation.  It  would  be  better  to  refrain  from 
using  the  cxprcssional  notation  for  this  kind  of  program. 

'Ihc  cxprcssional  loop  notation  is  also  limited  in  the  kind  of  loop  fragments  which  it  can  represent.  As 
described  above,  one  area  of  limitation  is  a  result  of  the  simple  notion  of  sequence  which  underlies  the 
notation.  This  makes  it  impossible  to  express  fragments  which  alter  die  order  of  elements  in  a  sequence  or 
which  merge  sequences. 

Ihc  only  facilities  available  for  creating  fragments  arc  the  meui  sequence  functions.  Fxpcricncc  has  shown 
that  these  are  capable  of  creating  a  wide  range  of  useful  fragments.  However,  there  arc  a  variety  of  plausible 
fragments  which  cannot  be  created'.  For  example,  ENUMERATES  always  creates  fragments  where  the 
termination  test  is  performed  at  the  start  of  each  cycle  of  the  loop.  It  is  not  possible  to  create  a  fragment 
where  the  termination  test  is  performed  at  the  end  of  each  cycle. 
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Another  example  of  a  fragment  that  cannot  be  represented  as  a  sequence  function  is  the  idea  of  doing 
output  to  a  report  file  as  discussed  above  in  conjunction  with  the  program  INVENTORY-REPORT.  An 
important  thing  to  notice  from  that  example  however,  is  that  LETS  makes  it  possible  to  combine  a  fragment 
like  this  one  w  ith  a  loop  in  expressional  notation,  litis  considerably  extends  the  domain  of  applicability  of 
the  notation.  A  vital  feature  of  this  is  th  it  the  notation  acts  to  protect  the  semantic  integrity  of  the  standard 
fragments  when  a  non-standard  fragment  is  added.  The  primary  w  ay  it  does  this  is  by  hiding  the  internal  stile 
variables  of  the  standard  fragments,  so  that  non-standard  fragments  cannot  modify  them. 

Another  more  fundamental  reason  why  the  expressional  loop  notation  may  not  be  appropriate  is  that  some 
other  paradigm  may  be  more  appropriate,  for  example,  consider  the  function  GCD.  Writing  it  as  a  recursive 
program  makes  it  very  easy  to  understand  because  the  structure  of  the  program  exactly  mirrors  the  structure 
of  live  standard  proof  of  correctness  for  die  algorithm.  No  iterative  rendition  would  be  as  clear. 

(defun  ged  (x  y) 

(cond  ((<  x  y)  (psetq  x' y  y  x) ) ) 

(let  ((r  (remainder  x  y))) 

(cond  ((zerop  r)  y) 

H  <9Cd  y  r))))) 

In  addition,  it  should  he  noted  that  unlike  some  looping  notations  the  expressional  notation  docs  not 
handle  anything  but  simple  loops.  For  example,  it  docs  not  support  multiple  entry  points  nor,  by  itself,  exits 
to  multiple  points. 


Efficient  Execution 

There  arc  two  principal  ways  in  which  the  expressional  notation  could  be  executed:  direct  execution ,  and 
conversion  to  iterative  loops.  Ilic  most  straightforward  way  would  be  to  just  implement  sequences  as  normal 
data  objects  and  the  sequence  functions  as  normal  functions.  Loop  expressions  could  then  he  evaluated  just 
like  any  other  expressions.  Ibis  direct  execution  approach  is  taken  by  API. |10|.  On  live  other  hand,  a 
compilation  process  can  be  used  to  convert  loop  expressions  into  ordinary  iterative  loops  which  operate  in  an 
clement  at  a  time  fashion.  This  conversion  approach  is  used  by  LETS  and  the  languages  Hibol  |12.13]  and 
Model  1 11). 

T  he  main  advantage  of  direct  execution  is  that  it  is  easy  to  implement.  In  particular,  it  is  very  easy  to  see 
how  it  directly  supports  the  expressional  metaphor.  The  main  disadvantage  of  direct  execution  is  that,  in 
comparison  with  ordinary  iterative  loops,  it  imposes  very  large  time  and  space  overheads. 

The  main  advantage  of  the  conversion  approach  is  Unit  it  is  capable  of  creating  very  efficient  code.  In  fact 
there  is  no  reason  in  principle  why  there  has  to  be  any  lime  or  space  overhead  at  all.  There  arc.  however,  two 
drawbacks  to  this  approach.  First,  the  conversion  process  can  be  quite  complex.  Just  how  complex  depends 
on  exactly  what  facilities  arc  supported  by  the  notation.  Second,  die  conversion  process  is  fundamentally 
related  to  the  element  at  a  time  viewpoint.  As  we  have  seen,  due  to  issues  like  termination  and  side-effects, 
this  viewpoint  cannot  be  hidden  from  die  user.  As  a  result,  the  user  has  to  keep  this  metaphor  in  mind  as  well 
as  the  simple  expressional  metaphor. 

When  designing  the  expressional  notation  it  was  Iclt  dial  the  issue  of  efficiency  could  not  be  ignored.  As  a 
result,  the  notation  was  designed  from  tin:  beginning  with  conversion  in  mind.  This  had  two  major  effects  on 
the  design.  First,  whenever  a  potential  feature  of  the  notation  would  have  unduly  complicated  the  conversion 
process  it  was  discarded.  Second,  the  element  at  a  lime  metaphor  was  introduced  as  an  explicit  part  of  the 
motivation  behind  the  notation.  These  precepts  resulted  in  a  notation  which  is  in  fact  relatively 
straight  forward  to  compile  into  efficient  code. 

The  ll'TS  macro  package  implements  the  notation  using  a  straightforward  set  of  macros  which  convert 
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each  expression  into  an  ordinary  loop.  No  explicit  representation  for  a  sequence  is  ever  required.  The 
compilation  process  is  described  in  detail  in  Appendix  B.  'Die  next  few  paragraphs  outline  the  major  features 
of  the  process. 

Hitch  sequence  function  is  represented  by  a  data  structure  specifying  some  initialization  computation  to 
perform  before  the  loop  begins,  some  inside  computation  to  perform  repetitively  on  each  cycle  of  the  Ittop, 
and  some  epilog  computation  to  perform  tiller  die  loop  terminates. 

A  composition  of  two  sequence  functions  "(A  (B  ...))"  is  compiled  by  combining  their  parts  together 
into  a  new  compound  sequence  function.  The  resulting  initialization,  insides,  and  epilog  arc  derived  by 
concatenating  the  corresponding  parts  of  B  and  A.  The  data  flow  from  B  to  A  is  implemented  by  data  flow 
from  the  inside  part  of  B  to  the  inside  part  of  A  in  the  new  compound  inside  part 

When  a  limp  expression  is  encountered,  it  is  first  parsed  in  order  to  locate  all  of  the  sequence  functions  in 
it.  lire  mcla  sequence  functions  arc  implemented  as  macros  which  take  their  functional  arguments  and  create 
an  appropriate  sequence  function.  As  part  of  the  parsing  process,  implicit  MAPS  introduction  and  other 
coercions  arc  introduced.  The  limp  expression  is  then  compiled  by  combining  all  of  the  sequence  functions  in 
it  together.  Once  this  has  been  done,  die  resulting  fragment  is  converted  into  an  actual  loop  with  the 
indicated  parts. 

Reluming  to  a  discussion  of  alternate  implementation  strategics,  it  should  be  noted  that  in  order  for  direct 
execution  to  be  used  with  the  cxprcssional  notation  the  notation  would  have  to  Ik  altered  in  several 
non-trivial  ways.  To  start  with,  the  notation  supports  potentially  infinite  sequences.  For  example,  inside  the 
typical  enumerator  is  a  generator  creating  an  infinite  sequence,  and  a  truncalor  cutting  this  down  to  finite 
length.  You  cannot  just  compute  the  entire  generated  sequence  before  truncating  it.  The  easiest  way  to  deal 
with  this  is  to  follow  the  lead  of  API.  and  simply  outlaw  generators  and  infinite  sequences,  allowing  only 
enumerators  of  finite  sequences.  However,  as  we  have  seen,  subsidiary  generators  can  be  very  convenient  in 
programs  such  as  01GITS-T0-NUMBER. 

A  much  more  fundamental  set  of  problems  arises  from  the  fact  that  the  element  at  a  time  metaphor  is 
fundamentally  incompatible  with  direct  execution.  Ilic  behavior  of  termination  and  side-effects  is  completely 
different  in  the  context  of  direct  execution.  In  many  situations  it  is  not  dear  whether  these  alternate  behaviors 
would  be  more  or  less  useful.  However,  with  regard  to  programs  like  INVENTORY-REPORT  which  are 
particularly  well  suited  to  the  dement  at  a  time  metaphor,  they  arc  quite  likely  to  be  less  convenient.  In  any 
case,  it  is  a  fundamental  change  and  much  of  the  nutation  would  have  lo  be  redesigned. 

There  has  been  a  lot  of  interesting  work  which  tries  to  chart  a  middle  course  between  the  simplicity  of  the 
direct  execution  approach  and  the  efficiency  of  the  conversion  approach.  One  way  in  which  this  has  been 
done  is  by  representing  sequences  explicitly,  but  without  trying  to  compute  the  elements  in  them  until  they 
arc  actually  needed.  This  can  be  done  explicitly  through  coroutines  |7],  or  implicitly  through  laz.y 
evaluation  (4.6].  In  l.ispMachinc  Lisp,  this  could  be  done  by  using  streams  to  represent  sequences.  Note  that 
this  delayed  evaluation  approach  is  capable  of  dealing  with  infinite  sequences  as  well  as  finite  ones.  Also  note 
that  from  the  point  of  view  of  what  gets  executed  when,  this  approach  entails  a  more  or  less  complete 
conversion  to  element  at  a  time  processing. 

Although  delayed  evaluation  is  more  efficient  (particularly  in  space)  than  direct  execution  in  many 
situations  it  is  still  much  less  efficient  than  complete  conversion.  Several  researchers  have  pursued  an 
interesting  mixed  mode  approach  which  provides  an  interpreted  implementation  where  sequences  are 
represented  explicitly  and.  in  addition,  provides  a  compiler  which  performs  conversions  to  eliminate 
intermediate  sequences  whenever  possible. 

Ihe  premier  example  of  this  has  been  the  work  on  compilers  for  API.  (2.5).  Optimizing  API.  compilers 
attempt  to  locate  array  expressions  where  the  arrays  are  being  used  merely  as  intermediate  sequences,  and 
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llien  eliminate  the  actual  computation  of  these  arrays.  When  an  expression  corresponding  to  live  kind  of 
simple  loop  representable  by  the  cxprcssional  notation  is  located,  then  it  is  easy  to  eliminate  the  intermediate 
arrays.  Wadlcr's  Listless  Transformer!  15]  pursues  a  similar  approach  for  compiling  a  Lisp-like  language.  It 
takes  programs  where  finite  intermediate  sequences  arc  represented  as  lists,  and  converts  them  into  programs 
where  these  intermediate  lists  do  not  actually  have  to  I5C  created.  The  resulting  programs  ran  then  be 
efficiently  compiled  by  normal  means. 

Unfortunately,  there  arc  several  inherent  problems  with  the  partial  conversion  approach.  First,  since 
direct  execution  of  unconverted  loop  expressions  must  be  supported,  the  notation  must  have  atl  of  the 
restrictions  outlined  above.  Second,  the  only  reason  to  pursue  partial  conversion  is  that  the  notation  supports 
features  which  cannot  be  practically  converted.  Unfortunately  this  raises  a  whole  new  problem  --  that  of 
identifying  what  parts  of  what  loops  can  be  converted.  In  addition,  steps  have  to  be  taken  to  interface  loop 
expressions  which  have  been  converted  with  those  which  have  not 

A  third  and  much  more  serious  problem  is  that  in  the  presence  of  side-effects,  conversion  is  not  a 
correctness  preserving  process.  The  reason  for  this  is  that  it  entails  a  radical  change  in  execution  order  from 
computing  each  sequence  as  a  unit  to  processing  several  sequences  an  element  at  a  time.  To  deal  with  this 
you  cither  have  to  refrain  from  converting  any  loop  containing  side-effects  (including  input/output)  or  you 
have  to  specify  that  such  loops  will  always  be  converted  and  require  the  user  to  think  in  tcrm>  of  the  element 
at  a  time  metaphor.  Note  that  the  latter  approach  cannot  be  taken  if  it  is  possible  for  the  user  to  write  a 
side-effect  containing  loop  which  cannot  be  converted. 

The  approach  taken  here  has  been  to  avoid  this  kind  of  problem  by  simplifying  the  notation  to  the  point 
where  complete  conversion  is  practical.  Ihc  languages  llibol  and  Model  support  somewhat  similar  notations 
(described  in  greater  detail  below)  which  arc  also  suitable  for  complete  conversion. 

Language  Independence 

All  of  the  discussion  above  was  couched  in  terms  of  Lisp,  and  die  initial  implementation  of  the 
cxprcssional  notation  has  been  done  for  Lisp.  However,  none  of  die  ideas  presented  here  are  inherently 
dependent  on  any  specific  language.  It  is  true  dial  it  is  particularly  easy  to  make  diis  kind  of  extension  to  the 
language  Lisp.  However,  by  modifying  its  compiler  this  could  be  introduced  as  an  extension  to  any  language. 
l;or  example,  you  could  add  the  cxprcssional  loop  notation  to  the  language  Ada  fl]  by  supporting  the  six  basic 
features  of  the  notation  as  follows: 

Sequence  Functions  -  As  in  the  Lisp  implementation,  calls  on  sequence  functions  would  look 
syntactically  exactly  like  calls  on  other  functions:  however,  they  would  be  handled  like  macros  in 
order  u>  create  loops  as  described  above.  A  set  of  basic  sequence  functions  would  be  provided  as 
part  of  the  standard  environment 

Sequences  -  A  new  data  type  SEQUENCE  OF  would  be  added.  This  could  be  used  to  specify  the  data 
types  of  variables  and  of  the  arguments  tu  sequence  functions. 

User  Definition  of  Sequence  Functions- A  new  kind  of  function  declaration  SEQUENCE  FUNCTION 
would  be  added.  Using  this,  sequence  functions  would  be  defined  exactly  like  ordinary  functions. 
These  would  be  the  only  functions  allowed  to  have  parameters  and/or  return  values  of  type 
sequence.  Similarly.  SEQUENCE  PROCEDURE  would  be  used  to  define  procedures  operating  on 
sequences. 

Meta  Sequence  Functions  -  ENUMERATES,  REDUCES,  etc.  would  be  provided  as  built-in  functions.  As  in 
the  I  isp  implementation,  Uicse  would  appear  syntactically  to  be  functions  taking  functional 
arguments;  however,  they  would  be  handled  by  the  compiler  essentially  as  macros. 
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I  Atop  Expression  Works  -  A  new  keyword  HI-'GIN  SEQUENCE  EXPRESSION  would  be  introduced. 
'ITiis  could  be  used  in  place  of  BEGIN  in  begin  blocks,  subprogram  bodies  etc.  Only  these  blocks 
would  be  allowed  to  have  variables  of  type  sequence.  Each  such  block  would  be  compiled  into  a 
single  loop. 

Coercions  *  Given  that  the  sequence  data  type  would  be  used  to  identify  all  of  the  variables  which  carry 
sequences,  various  coercions  could  be  supported  in  exactly  the  same  way  as  in  the  ).isp 
implementation. 

'Hie  following  examples  show  what  loop  expressions  would  look  like  in  Ada.  The  first  is  a  program  which 
takes  in  a  vector  of  digits  and  computes  the  corresponding  integer.  The  second  program  illustrates  the 
definition  of  a  sequence  function. 

type  1nt_vector  Is  array  (integer  range  <>)  of  Integer; 
type  Int.sequence  Is  sequence  of  Integer; 

function  digits_to_number_ada(d1g1ts:  int_vector)  return  integer; 
function  t imes_ten(x:  integer)  return  Integer; 

begin  return  x*10;  end; 
digit,  scale:  int_sequence; 
begin  sequence  expression 
digit  :■  Evector(dlgUs) ; 
scale  :*  generateS(t1mes_ten ,  1); 
return  Rsum(d1g1t*scale) ; 
end; 

sequence  function  Rsum(1nts:  Int.sequence)  return  Integer; 
begin  sequence  expression 
return  reduceS(+,  0.  Ints); 
end; 

Due  to  the  type  information  which  has  to  be  specified  and  the  fact  that  there  is  no  compact  representation 
for  literal  functions,  the  above  programs  arc  quite  verbose.  However,  they  are  identical  in  basic  structure  to 
their  Lisp  counterparts. 
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IV  -  Comparison  With  Other  Loop  Notations 

Consider  the  program  SUM-POSITIVE-EXPRESSIONAl  (reproduced  below)  which  was  used  as  an  example 
in  the  beginning  of  this  paper.  There  arc  many  different  computationally  equivalent  ways  to  represent  any 
given  loop.  All  of  these  representations  arc  capable  of  expressing  the  same  basic  looping  algorithm.  In  order 
to  evaluate  the  usefulness  of  these  representations,  we  must  look  at  other  characteristics  beyond 
expressiveness. 

(defun  sum-positive-exprasslonal  (vector) 

(Rsum  (Fgreater  (Evector  vector)))) 

The  paramount  property  required  of  a  looping  representation  is  undersnwdability  i.e.,  how  easy  is  it  to 
look  at  a  loop  and  determine  what  the  loop  is  computing.  Two  closely  related  properties  arc  also  of  great 
importance.  The  first  is  contimtibility  i.e.,  given  a  specification,  how  easy  is  it  to  build  up  a  loop  which 
satisfies  the  specification.  I  he  second  is  modifiability  i.e..  given  a  loop,  how  easy  is  it  to  change  it  in 
accordance  with  a  change  in  its  specification. 

The  key  idea  behind  the  expressionat  loop  notation  is  that  most  looping  algorithms  arc  built  up  out  of 
stereotyped  fragments  of  looping  behavior  and  therefore  loop  programs  arc  easier  to  understand,  construct, 
and  modify  if  these  fragments  arc  expressed  as  easily  identifiable  syntactic  units.  In  the  cxprcssional  notation, 
loop  fragments  are  represented  by  sequence  l'unctions.  Many  other  looping,  notations  have  methods  for 
representing  at  least  some  loop  fragments.  Discussion  of  these  methods  is  the  major  theme  of  the 
comparisons  below. 

Two  tilings  act  as  the  focus  for  the  following  sections.  The  first  is  the  loop  in  the  program  SUM- 
P0S1T  ive  -  exp  Ft  ESS  iona  L.  Kach  section  shows  how  the  loop  notation  being  discussed  could  be  used  to 
express  this  algorithm.  'Die  second  focus  is-thc  six  basic  features  of  the  cxprcssional  notation.  I’hc  sections 
arc  ordered  from  simple  constructs  which  have  very  few  of  these  features  to  languages  like  APT  and  Hibol 
which  embody  most  of  them. 


PROG  and  GO 

ITic  program  SUM-POStTIVE-GO  shows  how  our  example  loop  could  be  implemented  using  a  PROG  and 
GO.  The  program  is  not  very  easy  to  understand  because  PROG  and  GO  suggests  a  particularly  unfortunate  way 
to  think  about  the  loop,  namely  that  it  is  basically  a  straight  line  piece  of  code  which  is  converted  into  a  loop 
by  the  addition  of  a  GO.  'Ibis  notation  embodies  none  of  the  basic  features  of  the  cxprcssional  notation.  The 
key  idea  which  is  being  missed  by  this  way  of  thinking  is  that  straightforward  loops  like  this  one  arc  built  up 
out  of  standard  fragments  of  loops  and  not  out  of  standard  straight  line  fragments. 

(defun  sum-positive-go  (vector) 

(prog  (sum  1  end) 

(setq  sum  0) 

(setq  1  0) 

(setq  end  (1-  (array-length  vector))) 
l  (cond  ((>  i  end)  (return  sum))) 

(conri  ((plusp  (aref  vector  1)) 

(setq  sum  (+  sum  (aref  vector  1))))) 

(setq  1  (1+  1)) 

(go  l) ) ) 

Instead  of  highlighting  the  loop  fragments,  the  program  breaks  them  op  into  pieces  and  then  mixes  the 
pieces  together  l-'or  example,  the  enumerator  is  broken  up  into  three  pieces:  an  initialization  which  sets  the 
starting  value  tor  I.  a  termination  test  Dial  terminates  the  loop  after  the  last  index  is  produced,  and  a  repetitive 
step  which  increments  I  each  time  around  the'  loop. 
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;  just  as  difficult  to  see  how  the  fragments  interact  as  it  is  to  identify  the  fragments  themselves.  'ITte 
rator  and  the  filter  interact  by  sharing  the  variable  I.  In  contrast,  the  interaction  between  the  filter  and 
luccr  is  represented  by  embedding  part  of  the  reducer  inside  of  the  filter  CONO.  This  is  particularly 
ing  because  the  COND  looks  like  it  is  implementing  an  ordinary  straight  line  conditional  fragment.  One 
look  carefully  at  llic  surrounding  context  in  order  to  sec  that  this  is  not  the  ease. 

Iiough  the  above  points  have  been  presented  primarily  as  problems  of  understandability,  they  cause 
much  trouble  with  regard  to  constructibility  and  modifiability.  In  particular,  the  fact  that  the 
nts  arc  not  localized  means  that  neither  the  construction  nor  modification  processes  can  be  localized, 
reatiy  complicates  both  tasks.  Another  problem  is  dial  since  the  various  fragments  are  just  mixed 
:r,  dierc  is  no  support  for  keeping  them  semantically  separate.  One  must  be  particularly  careful  that 
icing  a  new  fragment  will  not  disturb  one  of  the  other  fragments. 

>lhcr  kind  of  problem  with  PROG  and  GO  as  a  notation  for  straightforward  loops  is  that  it  supports  a 
r  of  features  which  are  needed  only  in  complex  situations  and  which  obscure  simple  loops  by  cluttering 
p.  Two  examples  of  diese  arc:  the  fact  diat  PROG  supports  multiple  tags,  GOs  and  ROT  UR  Ns;  and  the  fact 
lllows  multiple  assignments  to  the  key  variables  involved.  These  features  arc  particularly  problematical 
c  even  when  dicy  arc  not  being  used,  you  have  to  look  very  closely  in  order  to  determine  that  dicy  are 
not  being  used.  In  the  example,  you  have  to  verify  that  there  is  only  one  tag,  one  GO,  and  one  RETURN 
it  dierc  is  only  one  assignment  to  each  of  die  critical  variables  in  die  loop  before  you  can  have  any 
:ncc  in  what  is  going  on. 

:rc  arc  algorithms  for  which  a  PROG  and  GOs  arc  particularly  appropriate.  For  example,  if  a  program 
lents  a  finite  state  automaton,  GOs  can  be  used  to  directly  model  the  transitions.  GOs  can  also  be  used  to 
lent  various  exotic  multiple  entry  and  multiple  exit  loops.  However,  it  is  generally  recognized  that  GOs 
cr  the  best  way  to  implement  simple  loops. 

Tail' Recursive  Style 

program  SUM-POSITIVE-RECURSIVE  is  written  in  tail  recursive  style.  Though  it  looks  very  different 
JM-POSITIVE-GO  it  specifics  essentially  exaedy  the  same  algorithm.  A  compiler  which  knew  about  tail 
in  would  produce  the  same  object  code  for  the  two  programs.  SUM-POSITIVE-RECURSIVE  is 
lat  easier  to  understand  because  much  of  the  verbiage  is  removed.  There  is  no  longer  any  possibility 
iplc  tags,  GOs,  or  RETURNS.  As  a  result  the  reader  does  not  have  to  worry  about  diem.  In  addition,  the 
t  each  value  changes  only  once  on  each  cycle  of  the  loop  is  easy  to  see, 

lefun  sum-positive-recursive  (vector) 

(sum-posl tlve-recursi vel  vector  0  0  (1-  (array-length  vector)))) 

efun  sum-posit Ive-recurslvel  (vector  sum  1  end) 

(cond  ((>  1  end)  sum) 

(T  (sum-posltlve-recurslvel  vector 

(cond  ((plusp  (aref  vector  1)) 

(+  sum  (aref  vector  1))) 

(T  sum)) 

(1+  1) 
end)))) 

PROG,  the  tail  recursive  style  suggests  a  particular  way  of  looking  at  a  loop.  Namely,  that  we  should 
17.C  die  task  at  hand  into  a  problem  that  can  be  recursively  reduced  a  step  at  a  time  to  a  problem  diat  is 
)  solve.  In  this  case  die  trivia]  problem  is  adding  up  the  positive  elements  of  a  sub-vector  of  length 
he  generalized  problem  is  adding  up  the  positive  elements  of  a  sub-vector  and  adding  this  to  an  initial 
um.  The  recursive  step  involves  adding  one  element  into  the  partial  sum,  and  reducing  the  size  of  die 
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si  ib- vector. 

'Ihcrc  are  loops  which  can  he  best  understood  by  looking  at  them  from  the  recursive  viewpoint.  However, 
this  program  is  not  one  of  them.  Ihe  problem  is  that  the  tail  recursive  style  is  no  belter  than  a  PROG  at 
highlighting  the  fragments  that  the  loop  is  composed  of.  As  above,  the  fragments  arc  broken  up  and  mixed 
together.  In  addition,  die  way  die  fragments  interact  is  still  unclear,  l-’or  example,  part  of  the  reducer  is  still 
nested  in  the  filter.  'I  he  "(T  SUM)”  clause  which  has  to  be  added  into  the  filter  COND  makes  dial  interaction 
even  less  dear  dian  in  die  PROG  above.  Like  PROG  and  GO,  the  tail  recursive  style  does  not  support  any  of  the 
features  of  the  exprcssional  notation. 


FOR 

flic  next  few  sections  describe  notations  which  begin  to  support  the  idea  of  a  sequence  function  (i.e., 
fragments  of  looping  behavior  as  identifiable  units).  They  do  not  however  support  any  of  the  other  features 
of  die  exprcssional  notation. 

Most  algorithmic  languages  have  looping  constructs  which  facilitate  the  construction  of  simple  loops.  A 
typical  example  . ;  diese  is  die  Ada  FOR  construct  flj.  The  Ada  program  SUM_POSITIVE_FOR  illustrates  the 
use  of  this  construct.  One  benefit  of  FOR  is  that  like  the  tail  recursive  style,  it  clearly  delimits  die  extent  of  the 
loop  and  makes  it  clear  that  there  is  no  exotic  control  flow  going  on  in  conjunction  with  the  loop. 
Unfortunately,  it  is  less  helpful  with  regard  to  the  data  flow.  'ITierc  is  no  easy  indication  that  each  of  the 
critical  variables  is  only  modified  once. 

type  1nt_vector  Is  array  (Integer  range  <>)  of  integer; 

function  sum_positive_for(vector :  1nt_vector)  returns  Integer  is 
sum:  integer; 
begin 
sum  :*  0; 

for  1  in  vector 'range  loop 
if  vector(1)>0  then 

sum  :*  sum+vector(1); . 
end  if; 
end  loop; 
return  sum; 
end; 

A  much  more  interesting  aspect  of  FOR  is  Uiat  it  explicitly  represents  one  of  die  fragments  --  the 
enumerator  of  integers  over  the  bounds  of  the  array.  This  explicit  representation  of  the  fragment  is 
particularly  useful  because  (unlike  die  FOR  constructs  in  most  other  languages)  the  Ada  FOR  construct  protects 
he  semantic  integrity  of  the  fragment  by  prohibiting  the  loop  counter  from  being  modified  inside  the  loop. 
\s  a  result  this  particular  fragment  is  easy  to  understand,  construct,  and  modify. 

Unfortunately.  FOR  is  only  capable  of  supporting  this  one  kind  of  enumerator.  Ihcrc  is  no  support  at  all 
or  any  of  the  other  fragments  in  the  loop.  They  arc  represented  using  straight  line  code  in  exactly  the  same 
lay  as  in  SUM-POSITIVE -GO.  As  a  result  fOR  is  really  not  that  much  of  an  improvement  over  GO  with  regard 
o  these  other  fragments. 
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Iterators  in  CLU 

I  he  language  Cl.U  |8|  has  extended  die  concept  behind  the  FOR  construct  so  that  it  can  represent  other 
enumerators  besides  integer  enumeration.  In  CLU  you  can  define  a  program  called  an  iterator  which  takes  in 
some  unitary  arguments  and  creates  a  sequence  of  objects.  Ihc  iterator  can  then  he  used  in  a  FOR  in  order  to 
enumerate  a  sequence  of  elements  to  be  processed  in  the  body  of  the  FOR.  Cl.U  provides  a  number  of 
standard  iterators  including  one  corresponding  to  EVECTOB.  As  an  illustration,  the  first  program  below  shows 
how  EVECTOR  could  be  defined  if  it  did  not  already  exist.  The  program  SUM_PQSITIVE_CLU  then  shows  how 
die  iterator  could  be  used. 

Evector  =  1te*r(a:  array[1nt])  yields(lnt) 
i:  int  :*  array[int]$low(a) 
end:  Int  array[1nt]$h1gh(a) 
while  1  <=  end  do 
yield(a[1]) 

1  :=  1  +  1 
end 

end  Evector 

sum_pos1t1ve_.clu  *  proc(a:  array[int])  returns(lnt) 
sum:  Int  :»  0 

for  e:  Int.  In  Evector(a)  do 

If  e  >  0  then  sum  :  =  sum  +  e  end 
end 

return(sum) 

end  sum_pos11.ive_clu 

Because  there  arc  no  restrictions  on  die  form  that  the  body  of  an  iterator  can  take  (for  example  there  is  no 
requirement  that  it  even  be  a  loop),  iterators  arc  more  general  Ulan  the  enumerators  presented  here. 
However,  this  power  has  drawbacks.  For  example,  it  makes  it  more  difficult  to  define  a  meta  sequence 
function  like  ENUMERATES.  In  addition,  it  would  be  difficult  to  treat  iterators  like  macros  and  compile  them 
inline  in  the  loops  which  use  them.  'ITic  current  CLU  compiler  implements  iterators  as  separate  procedures 
whic  h  return  one  element  of  the  sequence  every  time  they  arc  called. 

From  the  standpoint  of  understandabilily,  an  important  aspect  of  iterators  is  that  their  semantic  integrity  is 
protected  by  the  fact  that  they  encapsulate  their  own  state.  In  fact,  iterators  embody  die  logical  concept  of 
enumeration  fully  as  well  as  the  enumerators  presented  here,  (It  should  be  noted  dial  the  language 
Alphard  [14]  has  a  similar  construct  called  a  generator.)  Unfortunately,  neither  of  these  languages  provided 
any  support  for  any  fragments  other  than  enumerators.  As  a  result,  each  of  these  constructs  is  only  a  limited 
(though  significant)  improvement  over  simple  FOR  in  the  direction  of  supporting  fragments. 

Lisp  DO 

Another  variant  on  FOR  is  die  Lisp  DO  construct  This  construct  is  interesting  because  it  rccognir.cs  the 
existence  of  loop  fragments  other  than  enumerators  and  attempts  to  group  their  parts  more  closely  together. 
Kach  of  the  initial  lines  of  the  DO  is  capable  of  representing  a  loop  fragment.  For  example,  die  initialization 
and  repetitive  step  of  the  index  enumerator  arc  combined  together  in  die  first  line  of  the  DO  in  the  program 
SUM-POSITIVE 'DO. 
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(defun  sum-posltlve-do  {vector) 

(do  {(i  0  (1+  1)) 

(end  (l-  (array-length  vector))) 

(sum  0)) 

((>  i  end)  sum) 

(cond  ((plusp  (aref  vector  1)) 

(setq  sum  (+  sum  (aref  vector  1))))))) 

Unfortunately.  00  is  very  restrictive  in  the  way  it  can  represent  fragments,  l-or  example,  the  termination 
test  of  the  enumerator  has  to  he  specified  separately,  causing  the  enumerator  u>  be  less  conveniently 
represented  than  in  a  FOR.  In  addition,  there  is  no  gtHnl  way  to  represent  a  filter  at  all.  Going  beyond  this,  the 
interactions  between  the  fragments  have  to  be  represented  in  the  same  clumsy  ways  as  in  the  programs  above. 
For  example,  a  COND  still  has  to  be  used  to  express  the  interaction  between  the  filter  and  the  reducer. 

At  a  more  fundamental  level,  although  DO  makes  it  easier  to  write  loop  fragments  as  identifiable  units,  it 
docs  not  enforce  their  semantic  integrity.  For  example,  you  could  easily  put  an  assignment  to  I  in  the  body  of 
the  DO.  If  you  did  this  the  computation  involving  I  would  no  longer  he  an  index  enumeration.  This  would  be 
particularly  confusing  because  the  first  line  of  the  DO  would  still  look  like  an  ordinary  index  enumeration. 

All  in  all.  it  is  clear  that  the  various  FOR  and  DO  constructs  arc  quite  beneficial  because  they  make  it  easier 
to  locate  a  simple  loop,  and  to  verify  that  it  is  indeed  simple.  However,  although  these  constructs  point  in  the 
direction  of  explicitly  supporting  loop  fragments  they  do  not  do  this  in  either  a  very  thorough  way  or  a  very 
semantically  strong  way.  As  a  result,  they  arc  only  a  modest  help  in  tire  understanding,  construction,  and 
modification  of  loops. 

The  Lisp  Map  Functions 

'I he  l.isp  MAP  functions  arc  very  restricted  in  what  they  can  do.  For  example,  they  cannot  be  used  to 
express  the  algorithm  used  in  the  examples  above.  -However,  when  they  can  be  used  they  arc  very  compact 
and  easy  to  understand.  Kach  of  the  six  MAP  functions  is  an  abbreviation  for  a  particular  combination  of  loop 
fragments.  ITic  pair  of  equivalent  expressions  below  shows  the  fragments  corresponding  to  MAPCAR. 

(mapear  function  x) 

same  as:  (Rllst  (maps  ^'function  (El  1st  x ) ) ) 

l  vach  MAP  function  embodies  a  certain  set  of  fragments  and  protects  their  semantic  integrity.  If  these 
fragments  are  appropriate  to  tire  algorithm  at  hand,  then  the  use  of  the  MAP  function  leads  to  a  program  which 
is  easy  to  understand,  construct,  and  modify.  The  cxprcssional  loop  notation  is  designed  to  extend  the  basic 
idea  embodied  in  the  map  functions  to  a  wider  domain  of  programming. 

The  Lisp  Macro  LOOP 

ITie  l.isp  macro  LOOP  |3J  is  a  significant  improvement  over  the  constructs  presented  above  because  it 
recognizes  loop  fragments  of  all  kinds  as  full  fledged  constituents.  Consider  the  program  SUM-POSITIVE - 
LOOP.  In  this  program,  the  enumerator,  filter,  and  reducer  arc  each  represented  on  a  separate  line  in  the  loop. 
This  gives  a  program  which  is  much  easier  to  understand,  construct,  and  modify  than  the  ones  above.  A 
number  of  standard  loop  fragments  are  supplied  as  part  of  the  macro. 
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(dafun  sum-positive-loop  (vector) 

(loop  for  item  being  each  vector-element  of  vector 
when  (plusp  Item) 
sum  item)) 

In  addition  to  supporting  relatively  general  fragments  and  their  combination.  LOOP  supports  the  creation 
of  user  defined  fragments  of  all  kinds.  The  example  below  shows  how  one  could  define  VECTOR-ELEMENT  OF 
which  is  the  equivalent  of  the  sequence  function  EVECTOR. 

(define- loop-path  vector-element  Evector  (of)) 

(defun  Evector  (Ignore  variable  ignore  phrases  Ignore  Ignore  Ignore) 

(sublis  (list  (cons  'expr  (cadar  phrases)) 

(cons  'variable  variable) 

(cons  'vector  (gensym)) 

(cons. '  i  (gensym)) 

(cons  'end  (gensym))) 

’(((vector)  (1  0)  (end)) 

((setq  vector  expr) 

(setq  end  (1-  (array-length  vector)))) 

(>  1  end) 

(variable  (aref  vector  1)) 
nil 

(1  O*  1))))) 

Unfortunately,  LOOP  neither  develops  the  concept  of  a  sequence,  nor  the  analogy  of  treating  loop 
fragments  as  Junctions.  'Ibis  prevents  it  from  expressing  loops  as  sequence  expressions  in  analogy  with 
ordinary  unitary  expressions.  Instead,  LOOP  supports  a  keyword-based  syntax  which  specifies  both  the 
fragments  to  be  used,  and  how  they  arc  combined.  ’Hie  way  fragments  can  be  combined  is  rather  restricted 
because  it  is  lied  up  with  the  keyword  parsing  algorithm. 

In  addition,  the  LOOP  macro  has  a  body  part  (not  used  in  the  example  above)  just  like  the  body  of  a  DO. 
'Ibis  body  can  contain  arbitrary  compulation  -  there  is  no  attempt  to  protect  the  semantic  integrity  of  the 
individual  fragments  in  the  initial  part  of  the  LOOP. 

Another  problem  with  LOOP  is  that  the  facilities  it  provides  for  defining  the  equivalent  of  new  sequence 
functions  are  rather  cumbersome.  Unlike  the  exprcssional  notation,  there  is  nothing  corresponding  to  the 
meta  sequence  functions.  The  user  has  to  define  a  function  which  can  deal  with  parsing  parts  of  the  LOOP 
syntax  and  which  returns  a  list  of  six  pieces  which  arc  put  in  different  places  in  the  loop  being  constructed. 
Acting  together,  these  pieces  have  to  perform  the  actions  of  the  desired  sequence  function.  At  the  most  basic 
level,  this  is  quite  similar  to  what  happens  in  the  exprcssional  notation.  However,  it  seems  better  if  the  user 
docs  not  have  to  interact  with  the  system  at  tills  low  a  level. 


APL 

There  arc  several  programming  languages  which  support  what  arc  essentially  exprcssional  loop  notations. 
The  oldest  of  these  is  APL  (10].  It  is  interesting  to  note  that  there  is  no  reason  to  believe  that  the  developers  of 
APL.  had  anything  like  the  exprcssional  loop  notation  in  mind.  Rather,  they  were  just  seeking  to  provide  a  set 
of  very  useful  operations  on  arrays.  However,  a  style  of  writing  APL  has  evolved  where  sequences  are 
implemented  as  arrays. 

The  implementation  of  sequences  as  bona  fide  data  objects  automatically  supports  four  of  the  six  features 
of  the  exprcssional  notation  (i.e.,  sequences,  sequence  functions,  user  definition  of  sequence  functions,  and 
loop  expression  blocks).  As  illustrated  below,  both  sequence  ftmetions  and  the  vector  summing  algorithm  can 
be  very  compactly  represented  in  APL.  Note  that  since  sequences  arc  directly  represented  as  vectors,  there  is 
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no  need  for  the  function  EVECTOR. 

RESULT*FGREA'fER  VECTOR 
[1  ]  RESULT*! VECTORS ) /VECTOR 

V 

V  HESULT+RSUM  VECTOR 
[1]  RESULT** /VECTOR 

V 

V  SUM*SUMPOSI Tf VEAPL  VECTOR 

[  1  ]  SUM*RSUM( FGREATER ( VECTOR ) ) 

V 

API,  also  has  operators  similar  to  the  meta  sequence  functions.  For  example,  "function/  value"  is  the  same 
as  (REDUCES  function  init  value)  and  "functinn\value"  is  the  same  as  (SCANS  Junction  init  value).  (In  both 
eases  die  init  is  automatically  chosen  to  be  the  identity  element  under  function.)  Unfortunately,  user  defined 
functions  cannot  be  used  with  these  operators,  so  each  one  only  actually  corresponds  to  a  small  number  of 
built-in  sequence  functions.  AIM,  supports  the  notion  of  a  filter  in  a  more  general  way  than  it  supports 
RE  UCES  and  SCANS.  "( function ( value) )/ value"  is  the  same  as  (FILTERS  function  value).  This  operator 
(the  two  argument  form  of  /).  which  is  called  compression,  takes  in  two  vectors  and  creates  a  vector  of 
elements  from  the  second  vector  which  correspond  to  non-zero  elements  of  the  first  vector.  Any  arbitrary 
function  can  be  used  to  create  the  first  vector.  A  binary  function  rather  than  a  unary  one  is  used  in  the 
example.  (Note  that  compression  makes  a  shorter  vector,  rather  than  introducing  empty  elements.) 

API.  has  no  operators  corresponding  to  the  meta  sequence  functions  GENERATES.  ENUMERATES,  or 
TRUNCATES.  Since  sequences  arc  represented  as  arrays,  there  docs  not  have  to  be  any  equivalent  of  the 
sequence  functions  EVECTOR  and  RVECTOR.  Further,  since  arrays  arc  the  only  composite  data  structure 
supported  by  API.,  there  do  not  have  to  be  any  enumerators  or  reducers  which  deal  with  other  data  structures. 
Since  all  arrays  arc  finite,  there  need  not  be  any  generators  or  mmcators.  API.  docs  have  an  operator  (the 
index  generator  "  i It")  corresponding  to  ( ERANGE  1  N).  Note  that  the  fact  that  the  meta  operators  provided 
by  API.  arc  somewhat  limited  does  not  prevent  the  user  from  defining  any  kind  of  sequence  function  he 
desires  by  simply  using  more  primitive  constructs  to  write  the  appropriate  function  on  arrays. 

API.  also  supports  the  idea  of  implicit  MAPS  to  some  extent  Hvery  scalar  function  can  be  applied  to 
vectors  with  the  meaning  that  the  operation  is  to  be  applied  to  every  element  of  the  vector.  Also,  scalars  are 
coerced  to  vectors  wherever  necessary.  Both  of  these  processes  are  happening  in  the  expression  ( VECTOF>  0 ) 
above  which  takes  in  VECTOR  and  produces  a  vector  of  zeros  and  ones  which  indicate  which  elements  of 
VECTOR  are  greater  then  zero.  I  his  cannot  be  done  as  completely  as  with  the  cxprcssional  notation  presented 
here  because  there  is  no  mechanism  for  differentiating  between  arrays  which  arc  arrays,  and  ones  which  are 
intended  to  be  sequences. 

Ihcrc  arc  two  ways  in  which  API.  is  more  powerful  than  the  cxprcssional  notation  presented  here.  First,  it 
supports  a  number  of  operators  which  arc  much  more  powerful.  For  example,  it  has  a  number  of  operators 
which  rearrange  the  order  and  su  ucturc  of  an  array  such  as  reshape,  concatenation  (of  two  vectors),  expansion 
(the  inverse  of  compression),  reversal,  rotation,  and  grade  up  (sort).  It  has  complex  meta  operators  on  pairs  of 
arrays  such  as  outer  product  and  inner  product  which  produce  outputs  which  arc  not  the  same  shape  as  the 
inputs.  In  addition  to  all  this,  arrays  are  of  course  also  just  data  objects,  and  you  can  operate  on  them  as  such. 
You  can  retrieve  and  set  individual  elements  and  perform  arbitrary  computations. 

Another  way  in  which  API  is  more  powerful  is  that  while  sequences  are  analogous  to  vectors,  the  standard 
inlennediate  structure  in  API.  is  the  array.  The  fact  that  arrays  are  multidimensional  makes  them  a  more 
flexible  representation.  All  of  the  operators  above  can  be  applied  to  arrays  and  to  selected  pans  of  arrays 
producing  results  of  similar  or  dissimilar  shape. 
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The  powerful  features  provided  by  APL  make  it  possible  to  compactly  express  a  wide  variety  of  complex 
mathematical  algorithms  which  cannot  be  expressed  in  the  cxprcssional  notation  at  all.  l  or  these  algorithms, 
API.  has  the  virtue  of  easy  understandability,  constructibilily,  and  modifiability.  Unfortunately  API.  has 
several  drawbacks.  Hirst,  it  does  not  support  any  data  structures  other  than  numbers,  characters,  and  arrays. 
Second,  although  API .  supports  the  cxprcssional  metaphor  almost  completely,  it  docs  not  support  the  element 
at  a  time  metaphor  at  all.  Third,  due  to  that  fact  that  it  supports  such  complex  array  functions  and  the  fact 
that  it  rejects  the  element  at  a  time  metaphor.  API.  cannot  in  general  be  compiled  into  cUicicnl  code.  Fourth, 
APL's  approach  to  loops  is  embedded  into  a  somewhat  cryptic  and  forbidding  syntax.  Together,  these 
features  have  limited  API .'s  impact. 

The  cxprcssional  loop  notation  presented  here  eliminates  these  problems.  First,  it  can  handle  arbitrary 
data  structures.  For  example,  to  deal  with  a  new  aggregate  structure,  the  user  need  only  define  new 
enumerators  and  reducers  to  convert  the  aggregate  to  a  sequence  and  vice  versa.  Second  the  element  at  a  time 
metaphor  is  part  of  the  basis  for  the  notation.  'Hurd,  the  cxprcssional  loop  notation  deliberately  omits  all 
those  operations  on  sequences  which  would  make  it  hard  to  compile.  Fourth,  the  cxprcssional  notation  is 
designed  to  be  added  into  preexisting  languages  as  a  natural  extension  of  their  syntax.  One  need  not  learn  a 
new  language  and  environment  in  order  to  use  it 

The  Listless  T ransformer 

In  a  Lisp-like  language  one  could  decide  to  support  the  cxprcssional  metaphor  by  implementing 
sequences  as  lists.  Wadlcr[15]  has  implemented  an  interesting  prototype  system  (the  listless  transformer) 
which  is  capable  of  transforming  programs  containing  sequences  implemented  as  lists  and  eliminating  the 
actual  compulation  of  intermediate  lists.  The  loop  notation  supported  by  his  system  is  at  heart  essentially 
identical  to  API.  with  lists  substituted  for  arrays.  The  target  of  his  system  is  a  Lisp-like  language  called 
Iswim  (IS].  The  example  below  shows  how  sequence  functions  can  be  defined  and  used  in  this  language. 

def  Evsctor(v)  • 

Evectorl(v,  0,  length(v)) 

where  rec  Evectorl(v,  -1,  end)  » 

If  1>end  then  nil 

else  cons(aref(v,  i),  Evectorl(v,  1+1,  end)) 

def  rec  Fgreater(xs)  ■ 
case  xs  of 
nil  «>  nil 

cons(x  .rest)  «>  If  x>0  then  cons(x,  Fgreater(rest)) 

else  Fgreater(rest) 

def  Rsum(xs)  « 

Rsuml(xs,  0) 

where  rec  R$uml(xs,  total)  • 
case  xs  of 
nil  «>  total 

cons(x,  rest)  =>  Rsuml(rest,  total+x) 

def  suw'posltlve-llstless(v)  « 

Rsum( Fgreater( Evector(v) ) ) 

It  is  not  clear  whether  any  meta  sequence  functions  arc  supported:  however,  they  would  be  easy  to 
implement  as  macros.  In  any  case,  die  user  can  implement  any  sequence  function  he  desires  be  defining 
arbitrary  functions  on  lists.  Iswim  is  a  typed  language  and  coercions  like  implicit  introduction  of  MAPS  can  be 
supported. 

Like  API.,  Wadlcr's  notation  is  more  powerful. than  the  notation  described  here  in  that  it  supports 
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arbitrarily  complex  sequence  functions  and  sequences  can  be  multi-dimensional  sequences  of  sequences.  It 
goes  beyond  API .  in  being  able  to  deal  with  arbitrary  data  structures. 

Wadlcr's  notation  also  shares  AM.’s  greatest  weaknesses.  It  dues  not  support  the  element  at  a  lime 
metaphor.  In  addition,  due  to  the  fact  that  arbitrarily  complex  sequence  functions  arc  allowed,  it  cannot  be 
efficiently  compiled  in  the  general  ease.  It  also  shares  the  problem  that,  since  it  is  not  oriented  toward  the 
element  at  a  time  metaphor,  loops  involving  side-effects  cannot  be  efficiently  compiled. 

Coroutines 

Another  language  which  supports  most  of  the  cxprcssional  metaphor  is  the  coroutine  language  of  Kahn 
and  MneQueen  [7J.  They  have  suggested  using  parallel  processes  (coroutines)  in  order  to  represent 
computations  communicating  via  one  way  channels  (sequences).  In  their  approach,  unbounded  sequences  are 
implemented  as  real  data  objects  which  arc  passed  an  element  at  a  time  through  channels  between  processes 
executed  in  parallel.  'I he  code  below  shows  one  way  the  vector  summing  algorithm  could  be  implemented  in 
their  system.  Kach  sequence  function  is  defined  as  a  separate  process.  ITicse  processes  can  have  ordinary 
(unitary)  inputs  (c.g.,  die  VECTOR  input  of  EVECTOR)  and  outputs  (c.g..  the  return  value  of  RSUM).  'Ihey  can 
also  have  channel  (sequence)  inputs  (c.g..  the  CHANNEL  1  argument  of  FGREATER)  and  outputs  (c.g.,  the 
CHANNEL  output  of  EVECTOR).  An  element  is  retrieved  from  a  channel  by  the  function  (GET  channel).  An 
element  can  be  put  into  a  channel  by  the  function  (PUT  Hem  channel) .  In  order  to  use  the  sequence 
functions,  they  arc  combined  together  in  an  expression  as  in  the  function  SUM-POSITIVE-COROUTINE.  This 
expression  is  placed  in  a  DOCO  form  which  causes  the  three  processes  to  be  executed  concurrently. 

process  Evector  vector  =>  channel; 
wars  1; 

1  ->  1; 

repeat 

put(1,  channel); 

Increment  1; 

until  1>upper-bound(vector) ; 
put(done,  channel) 
endprocess; 

process  Fgreater  In  channell  *>  channel?; 
vars  n; 
repeat 

get(channell)  ->  n; 

If  n»done  or  n>0  then  put(n,  channel2)  close 
until  n=done 
endprocess; 

process  Rsum  in  channel  *>  sum; 
vars  n.  sum; 

0  ->  sum; 
repeat 

get(channel)  ->  n; 

If  not(n=done)  then  sum+n  ->  sum  close 
until  n*done; 
return(sum) 
endprocess; 

process  sum-positive-coroutine  vector 
start  doco  Rsum( FgreaterfEvector  vector))  closeco 
endprocess; 

The  language  of  Kahn  and  MacQuecn  supports  neither  meta  sequence  functions,  nor  automatic  coercions. 
However,  they  could  be  added  if  desired.  It  is  interesting  to  note  that,  unlike  AM.,  coroutines  directly 
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support  the  element  at  a  time  metaphor. 

'Che  coroutine  approach  is  more  powerful  than  the  cxprcssional  notation  along  a  different  dimension  from 
AW..  Fach  process  is  a  truly  independent  parallel  process.  One  aspect  of  this  is  that  sequences  can  really  be 
infinite.  In  addition,  it  is  possible  for  one  process  to  terminate  without  forcing  the  other  ones  to  terminate  and 
processes  can  dynamically  spawn  whole  networks  of  other  processes,  'litis  makes  it  possible  to  express  inodes 
of  compulation  which  cannot  be  conveniently  expressed  with  any  of  the  other  notations  discussed  here. 
However,  this  brings  with  it  a  certain  overhead.  In  lltc  example  above,  the  special  token  DONE  is  passed 
around  between  the  processes  so  that  the  termination  of  the  EVCCTOR  process  will  trigger  die  termination  of 
the  other  processes. 

The  key  drawback  of  the  coroutine  approach  is  that  it  is  not  clear  how  it  can  be  compiled.  Since  it 
supports  die  element  at  a  time  metaphor,  certain  logical  obstacles  arc  removed;  however,  like  API.,  it  supports 
the  definition  of  arbitrarily  complex  sequence  functions.  Going  beyond  this,  given  that  the  coroudnc 
notation  is  capable  of  expressing  arbitrary  parallel  computations,  one  would  expect  that  it  will  be  extremely 
difficult  to  write  an  optimizing  compiler  which  reliably  detects  groups  of  processes  which  interact  merely  as 
simple  loops.  However,  without  such  a  compiler,  the  coroutines  impose  an  unacceptable  overhead  on  the 
execution  of  simple  loops. 

'(Tie  cxprcssional  loop  notation  presented  here  is  based  on  ideas  very  similar  to  the  coroutine  notation; 
however  it  is  restricted  so  diat  it  is  trivial  to  compile.  The  intention  is  to  use  the  cxprcssional  notation  to 
represent  simple  loops  while  reserving  die  coroudnc  notation  for  those  situations  where  its  greater  power  is 
required. 


Hibot  &  Model 

The  language  Hibol  [12.13]  is  the 'oldest  language  which  both  supports  the  idea  of  a  sequence  and  is 
completely  compilable.  It  is  a  very  high  level  business  data  processing  language  based  tin  the  concept  of  a 
flow  (which  is  basically  equivalent  to  a  sequence).  It  is  very  strongly  oriented  toward  die  element  at  a  time 
metaphor  and  relies  heavily  on  the  concept  of  the  implicit  introduction  of  HAPS.  Hie  body  of  each  Hibol 
program  is  a  nonprocedural  set  of  expressions  specifying  the  computations  on  typical  sequence  elements. 

The  program  SUM_P0SITIVE_HIB0L  computes  the  sum  of  die  positive  elements  in  a  file.  (The  only 
aggregate  data  type  supported  by  Hibol  is  a  file.)  The  language  provides  a  few  standard  sequence  functions 
(c.g.,  the  operator  SUM  in  the  program  below).  In  addition,  die  operator  IF  implements  the  mcla  sequence 
function  FILTERS.  These  facilities  make  it  possible  to  specify  the  body  of  SUM_POSITIVE_HIBOL  as  a  simple 
expression.  ITic  DATA  DIVISION  part  of  the  program  describes  the  files  accessed  by  the  program  in  a  format 
very  similar  to  Cobol. 
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/•  the  program  sum_po$1t1ve_h1bo1  •/ 
data  division 
key  section 
koy  Index 

field  type  Is  Integer 
input  section 
file  vector.ltem 
key  Is  Index 
type  Is  Integer 
output  section 
file  sum_pos1t1ve 

type  Is  Integer 

computation  division 

sum_pos1t1ve  Is  (sum  of  (vector_1tem  if  vector_1tem  >  0)) 

Hibol  is  more  powerful  than  the  cxprcssional  notation  in  that  flows  arc  multidimensional  objects  like 
arrays  where  each  level  of  index  is  an  alphanumeric  key  rather  titan  a  number.  The  Hibol  operators  can  be 
selectively  applied  to  specific  dimensions  of  a  flow.  A  set  of  defaulting  mechanisms  make  it  possible  to 
specify  a  simple  program  like  the  one  above  without  having  to  explicitly  specify  which  dimensions  operators 
arc  being  applied  to.  However,  the  operations  which  can  be  applied  to  flows  have  been  carefully  selected  so 
that  all  (low  expressions  can  be  compiled  into  efficient  loop  code. 

The  Hibol  compiler  clearly  shows  that  an  cxprcssional  looping  notation  can  be  straightforwardly  compiled 
even  if  it  supports  multidimensional  sequences.  Nevertheless,  when  designing  the  cxprcssional  loop  notation 
presented  here  it  was  decided  to  omit  this  feature  for  two  reasons,  f  irst,  it  was  judged  that  the  frequency  of 
its  use  would  not  justify  the  extra  complexity  of  supporting  it.  Second,  when  a  loop  algorithm  becomes 
complex  enough  that  the  user  is  forced  to  specify  which  dimensions  operators  arc  being  applied  to,  the 
syntactic  mechanisms  required  cause  the  resulting  expressions  to  begin  to  lose  the  virtue  of  easy 
understandability. 

From  the  point  of  view  of  this  discussion  the  primary  weakness  of  Hibol  is  that  it  docs  not  provide  very 
much  support  for  the  cxprcssional  metaphor.  First,  it  provides  a  few  built  in  sequence  functions,  but  docs  not 
allow  the  user  to  define  new  ones.  Note  that  files  arc  the  only  aggregate  data  structure  supported  by  Hibol, 
and  that  enumeration  and  reduction  of  files  occurs  implicitly.  Second,  it  supports  only  two  meta  sequence 
functions:  MAPS  (introduced  only  implicitly)  and  FILTERS  (the  form  IF).  Implicit  nested  loops  can  be 
specified  but  there  is  no  notion  of  an  explicit  looping  block. 

As  discussed  above,  the  cxprcssional  loop  notation  presented  here  addresses  these  problems  because  it  can 
deal  with  arbitrary  data  structures,  because  it  supports  the  creation  of  user  defined  sequence  functions,  and 
because  it  is  intended  to  be  embedded  in  a  language  which  supports  standard  control  flow  constructs.  The 
cxprcssional  notation  being  presented  here  could  be  looked  at  as  taking  some  of  the  key  ideas  embodied  in 
Hibol  and  separating  them  out  from  the  business  data  processing  language  context  of  Hibol  in  a  form  in 
which  they  can  be  conveniently  added  into  other  languages. 

More  recently,  another  language  has  been  developed  which  is  very  much  like  Hibol.  This  language 
(Model  1 11))  is  based  on  the  same  idea  of  a  multidimensional  sequence,  and  is  also  primarily  intended  for 
business  data  processing  applications.  It  is  somewhat  more  powerful,  and  has  a  somewhat  wider  range  of 
features,  but  at  the  level  of  this  discussion  it  is  essentially  identical  to  Hibol.  It  is  fully  compilable  and  has  the 
same  basic  advantages  and  disadvantages.  It  serves  as  yet  another  cxjunple  that  the  idea  of  a  sequence  appears 
in  many  different  forms  in  many  different  languages. 
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Appendix  A:  The  Compilation  Process 

ITic  first  section  in  this  appendix  describes  some  assumptions  which  the  macro  expansion  process  makes 
about  die  form  of  die  loop  expressions  to  be  processed.  Hie  user  must  be  careful  to  ensure  diat  these 
assumptions  are  satisfied.  The  rest  of  the  sections  discusses  the  actual  macro  expansion  process  in  detail.  This 
discussion  is  intended  to  function  both  as  detailed  documentation  for  die  actual  program,  and  as  a  guide  to 
anyone  who  wishes  to  implement  a  similar  system. 

The  compilation  process  revolves  around  a  data  structure  representing  the  key  information  about  a 
fragment  of  looping  behavior.  I;ach  fragment  data  structure  contains  all  of  the  information  needed  to  create  a 
loop  corresponding  to  a  sequence  function,  and  information  about  the  inputs  and  outputs  of  die  sequence 
function.  A  call  on  a  sequence  function  is  represented  as  an  application  of  a  fragment  data  structure  to  a  list 
of  arguments.  The  process  of  combining  several  sequence  functions  together  into  a  single  loop  proceeds  by 
combining  together  the  fragment  data  structures  corresponding  to  them. 

Given  a  program  dial  contains  one  or  more  loop  expressions,  macro  expansion  will  proceed  normally  until 
the  outermost  macro  in  one  of  these  loop  expressions  is  encountered.  At  that  lime,  die  L  ETS  macro  package 
immediately  locates  all  of  die  inner  loop  macros  in  die  expression  and  constructs  a  loop  combining  diem  all 
together.  Macro  expansion  then  continues  normally  until  another  loop  expression  is  encountered. 

Iho  process  of  converting  a  loop  expression  into  a  loop  occurs  in  several  steps.  After  locating  an 
expression,  all  of  the  calls  on  sequence  functions  in  it  are  converted  into  applications  of  fragment  data 
structures.  Similarly,  calls  on  meta  sequence  functions  arc  converted  into  applications  of  fragment  data 
structures  built  out  of  the  functional  arguments  passed  to  the  meta  sequence  functions. 

Once  everything  has  been  reduced  to  an  application,  the  result  is  parsed  in  order  to  located  nested  loops. 
Any  nested  loops  arc  isolated  and  processed  separately.  A  second  phase  of  parsing  then  performs  coercions 
such  as  the  automatic  introduction  of  HAPS  and  AT-END.  The  resulting  group  of  applications  is  combined 
together  into  a  single  fragment  data  structure.  This  structure  is  then  converted  into  a  single  loop. 

As  an  example,  the  following  shows  the  code  which  is  produced  Tor  the  loop  in  the  program  SUH- 
POSITIVE-EXPRESSIONAL. 

(Rsum  (Fgreater  (Evactor  vector))) 
becomes :( prog  T  (1  end  num  sum) 

(setq  1  0) 

(setq  end  (1-  (array-length  vector))) 

(setq  sum  0) 

L  (cond  ((>  i  end)  (go  E))) 

(setq  num  (aref  vector  1)) 

(cond  ((>  num  0)  (setq  sum  (+  sum  num)))) 

(setq  1  (1+  1)) 

(go  L) 

E  (return-from  T  sum)) 
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Interaction  With  Other  Macros 

There  arc  two  important  (and  unfortunate)  restrictions  on  the  way  in  which  the  LETS  macro  package  can 
be  used  which  stem  from  the  fact  that  the  package  does  not  have  any  special  knowledge  of  either  system  or 
user  defined  fexprs  or  macros.  Ihcse  restrictions  could  be  removed  if  more  knowledge  was  built  into  the 
compilation  process. 

The  first  restriction  is  that  in  a  loop  expression,  every  list  whose  CAR  is  one  of  the  loop  macros  must  l>e  a  call 
on  that  macro.  Ihe  list  will  be  macro  expanded  and  combined  into  the  loop.  To  avoid  running  afoul  of  this 
assumption,  you  should  never  use  the  name  of  one  of  these  macros  as  the  name  of  a  variable.  Ihe  only  place 
where  this  restriction  docs  not  apply  is  inside  of  quoted  lists. 

The  second  restriction  is  that  for  each  variable  name  in  the  argument  list  of  a  OEFUMS,  bound  by  LET  S.  or  in 
the  lambda  list  of  a  literal  lambda  expression  passed  to  a  meta  sequence  junction,  every  occurrence  of  that 
symbol  in  the  body  must  be  an  instance  of  a  reference  to  that  variable.  Ihe  function  SUBST  will  be  used  to 
rename  this  variable  when  necessary  to  avoid  name  conflicts.  Ihe  two  main  ways  dial  trouble  could  arise  is  if 
you  use  a  variable  name  which  is  die  same  as  a  function  name,  or  if  you  rebind  the  variable  name  in  some 
inner  scope.  Note  that  you  cannot  even  use  the  variable  name  in  a  quoted  list 

Another  kind  of  restriction  arises  from  die  fact  that  the  LETS  macro  package  does  all  of  its  processing 
without  expanding  any  other  macros.  As  a  result  of  this,  only  lists  whose  CARs  arc  one  of  the  loop  macros  are 
albwetl  to  expand  into  a  loop  fragment  (as  opposed  to  a  complete  loop).  In  particular,  an  ordinary  macro 
cannot  expand  into  code  which  is  supposed  to  be  a  loop  fragment.  T  his  will  not  work  because  the  macro  will 
not  be  expanded  until  after  the  loop  it  is  in  has  already  been  completely  constructed.  Note  that  it  is  all  right 
for  a  macro  to  contain  a  complete  loop  expression  which  will  be  converted  into  a  ioop  as  a  whole  by  itself. 
The  appropriate  way  to  make  macros  describing  loop  fragments  is  to  use  OEFUUS.  For  example,  compare  the 
following  two  definitions  of  a  loop  fragment  which  enumerates  die  cars  of  the  elements  of  a  list.  Only  the 
second  one  will  work. 

(defmacro  car.-El  Ist-Puggy  (Input) 

(list  'car  (list  ‘Ellst  Input))) 

(defunS  car-Ellst  (Input) 

(car  (Ellst  Input))) 

Another  ci  ascqucnce  of  the  fact  that  LETS  does  extensive  processing  before  other  macros  arc  expanded  is 
that  you  cannot  nest  one  of  the  exprcssional  macros  inside  a  cal!  of  a  macro  that  looks  inside  of  its  argument. 
For  example,  even  assuming  that  you  define  a  SETF  property  for  EL  1ST,  you  cannot  write 
"( SETF  ( ELIST  L)  X)".  Ihe  problem  is  diat  since  the  loop  macros  arc  expanded  first,  SETF  will  never  get 
to  see  the  ELIST.  Also  note  that  instances  of  loop  macros  arc  usually  replaced  by  variables  in  die  resulting 
loop.  However,  you  can  say  things likethe  following  "(SETF  (CAR  (ELIST  L))  X)" because  the  SETF  docs 
not  need  to  look  at  the  argument  of  the  CAR. 

The  Representation  for  a  Fragment 

Loop  fragments  arc  represented  internally  by  die  following  structure: 

(S-frag  name  ((arg  type  .  info)  ...) 
icode  codel  code 2  pcode  uende) 

'Ihe  name  part  of  the  form  is  used  for  producing  more  understandable  error  messages.  It  records  what 
macro  generated  the  fragment.  I  he  second  part  of  the  form  is  a  list  of  argument  descriptors.  The  symbol  arg 
is  die  name  of  die  argument.  Kvery  internal  use  of  the  argument  is  represented  by  that  symbol.  Ihcrc  is  one 
argument  declaration  for  every  input,  output,  and  auxiliary  variable  used  by  the  ftagmcni.  I  he  order  of  die 
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declarations  is  used  to  match  the  inputs  up  with  parameters  when  the  fragment  is  used.  The  symbol  arg  is 
created  by  GENSYM.  It  is  guaranteed  to  be  unique  and  occur  only  in  this  single  fragment  As  a  result, 
renaming  docs  not  have  to  be  used  when  fragments  arc  combined  together.  If  the  fragment  is  copied,  then 
the  args  iirc  renamed  by  using  SUBST.  As  a  result,  every  internal  instance  of  the  symbol  must  correspond  to  a 
use  of  the  variable. 

Note  that  free  variable  inputs  and  outputs  are  not  mentioned  in  the  argument  descriptors.  Rather,  they 
•ire  just  referred  to  in  die  body  of  die  fragment  where  appropriate.  Due  to  die  fact  that  die  order  of  execution 
in  a  loop  expression  is  preserved,  things  work  out  all  right  when  fragments  are  combined  together  without  the 
system  having  to  dike  any  explicit  action.  In  fact,  the  system  ignores  the  presence  of  free  variables  entirely. 

Pitch  argument  has  a  type  which  is  one  of  the  following  symbols:  UI,  SI.  UO,  SO,  UP,  SP.  A,  and  F.  These 
symbols  tire  built  up  out  of  the  following  code  letters.  'Ihc  use  of  some  of  die  code  letters  requires  that 
additional  information  be  supplied  in  the  info  field. 

U  -  UNITARY  -  This  is  an  ordinary  1  ,isp  object.  Inputs  are  given  a  value  before  die  entire  fragment  is 
evaluated,  and  outputs  pass  out  their  final  value  when  the  whole  evaluation  is  over. 

S  -  SEQUENCE  -  'Ihis  specifies  that  the  argument  is  a  sequence.  The  variable  holds  successive 
elements  of  this  sequence.  On  each  cycle  of  die  loop.  c;tch  sequence  input  is  given  a  new  value 
before  the  first  time  it  is  read,  and  the  final  value  of  each  sequence  output  is  exported  out  of  the 
fragment 

I  -  INPUT  -  This  is  an  input  object  passed  in  via  nesting  in  argument  position. 

0  -  OUTPUT  -  This  is  an  output  passed  out  through  the  return  value.  There  can  be  more  than  one 
return  value  in  which  case  their  order  specifics  which  is  which  in  a  MULTIPLE-VALUE. 

A  -  AUX  -  This  is  an  internal  auxiliary  variable.  It  must  be  unitary.  If  the  info  field  is  non-NIL,  it 
indicates  dial  this  variable  was  specified  by  die  user  and  must  be  retained  in  the  final  loop  produced. 

P  *  OPTIONAL  -  This  is  an  optional  input.  When  the  definition  is  applied  it  is  converted  into  either 
an  input  or  an  aux.  The  info  field  contains  an  initializing  expression  to  use  when  a  parameter  value 
is  not  supplied  for  diis  argument. 

F  -  FLAG  -  'ITiis  is  an  auxiliary  flag  used  in  filtered  computations.  The  filtered  sequences  themselves 
arc  carried  in  separate  variables.  The  info  field  is  a  list  of  all  of  die  free  variables  and  return  values 
which  are  under  the  control  of  this  filter  flag. 

The  remainder  of  the  fragment  specifics  the  computation  to  be  performed.  The  icode  is  a  list  of  zero  or 
more  expressions  which  arc  executed  exactly  once  just  before  die  repetitive  part  of  the  loop  is  executed.  The 
node  cannot  refer  to  any  sequence  arguments.  It  can  read  only  unitary  inputs.  It  can  write  any  aux.  or  unitary 
output.  Its  effect  is  to  give  initial  values  to  variables.  T  ypically,  every  unitary  output  is  given  some  default 

Vu'f'iC. 

The  auic I  and  codc2  arc  the  repetitive  body  of  the  fragmenL  They  arc  the  only  places  where  sequence 
arguments  can  be  referred  to.  Both  of  these  arc  lists  of  zero  or  more  expressions.  Both  of  them  arc  executed 
on  every  cycle  of  the  loop  and  can  read  sequence  values.  The  code  I  (but  not  die  coJe2)  can  write  sequence 
values. 

There  arc  two  different  slots  here  because  of  die  following  property.  All  the  code/  parts  of  all  the 
fragments  being  used  will  be  executed  before  all  of  the  codcJ  parts.  This  gives  you  control  over  what  is  going 
on.  In  particular  all  terminations  arc  placed  in  code!  parts.  As  a  result,  you  can  depend  on  the  fact  that  the 
code  2  will  not  be  executed  on  die  cycle  where  die  loop  terminates.  (The  code  l  may  he.)  If  there  is  a  filter 
producing  some  of  the  inputs  read  by  code  I  or  by  aidrj  dien  both  will  be  c\aluaicd  only  on  those  cycles 
w  here  all  of  the  filtered  inputs  arc  available. 
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urinations  arc  represented  by  putting  a  COND  with  DONE  in  one  of  its  branches  in  the  code!.  Filters  are 
nted  by  using  the  form  (S-IF  flag- test,  actions)  in  the  code I.  The  effect  of  the  filter  is  obtained  by 
g  sequence  outputs  in  the  actions  and  making  the  appropriate  flag  argument  declarations. 
pcode  is  a  list  of  zero  or  more  expressions  which  is  executed  exactly  once  after  the  loop,  if  it  terminates 
ly.  It  cannot  refer  to  any  sequence  quantities.  Its  purpose  is  to  perform  epilog  computations  involving 
[ary  outputs. 

ucode  is  a  list  of  zero  or  more  expressions  which  is  executed  in  an  UNWIND- PROTECT  wrapped  around 
p  eventually  produced.  It  cannot  refer  to  any  sequence  quantities.  Its  purpose  is  to  perform  epilog 
ations  involving  the  unitary  outputs  which  must  be  performed  no  matter  how  the  loop  is  terminated, 
following  examples  illustrate  die  fragment  representation.  The  first  corresponds  to  the  sequence 
n  RUST.  Note  the  use  of  some  pcode  in  order  to  reverse  the  list  CONSed  up.  The  second  corresponds 
N6E.  Note  the  use  of  an  optional  parameter  BY.  the  presence  of  a  terminator,  and  that  the 
?n  tat  ion  of  the  state  is  placed  in  code!  so  diat  it  will  not  be  done  on  die  cycle  on  which  the  loop 
ites.  The  final  fragment  corresponds  to  FGREATER.  Notice  that  die  computation  of  die  filter  flag  is 
;d  from  the  S-IF  which  specifics  what  values  arc  controlled  by  diat  flag. 


;-frag  'Rlist  ((Item  SI)  (result  UO)) 
((satq  result  nil)) 

((setq  result  (cons  Item  result))) 

0 

((setq  result  (nreverse  result))) 

0) 


i*frag  ’Erange  ((state  UI)  (end  UI)  (by  UP  .  1)  (Int  SO)) 

0 

((cond  ((>  state  end)  (done)))  (setq  Int  state)) 

((setq  state  (+  state  by))) 

0 

0) 

-frag  ’Fgreater  (Mnt  SI)  (limit  UP  .  0)  (flag  F  big)  (big  SO)) 

0 

((setq  flag  (>  Int  limit))  (S-lf  flag  (setq  big  Int))) 

0 

0 

0) 


if  the  internal  macro  processing  revolves  around  fragments  represented  in  the  above  form.  They  are 
;d  together  into  larger  and  larger  fragments  and  then  converted  into  normal  loop  code. 


Sequence  Functions 

form  (S-APPLY  outs  fragment  parameter...)  is  used  to  apply  a  fragment  to  a  group  of  parameters, 
t  field  indicates  what  variables  (if  any)  the  outputs  arc  being  assigned  to.  T  indicates  dial  the  outputs 
being  returned.  Sequence  functions  arc  merely  macros  which  expand  into  S-APPLYs  of  fragments, 
lustrated  by  die  following  pair  of  expressions.  Note  that r  ‘•eh  time  die  fragment  is  instantiated,  the 
ts  in  it  are  renamed  so  that  there  cannot  be  any  name  clashes. 

(Rl 1st  x) 

it  as:  (S-apply  T  (S-frag  'Rlist  ...)  x) 

•nly  interesting  thing  which  happens  when  a  sequence  function  is  expanded  into  an  S-APPLY  is  the 
t  of  optional  arguments.  If  a  parameter  is  provided,  then  the  argument  list  of  the  fragment  is 
so  that  the  argument  is  specified  to  be  a  normal  input.  If  no  parameter  is  supplied  then  the  argument 


lie  Compilation  Process 


*52* 


Waters 


.  converted  into  an  aux  and  the  initializing  expression  is  used  to  give  the  argument  a  value  either  in  the  icode 
if  it  is  unitary)  or  in  the  code l  (if  it  is  a  sequence  value).  Note  that  this  expression  is  evaluated  inside  the 
ragment  and  can  refer  to  all  of  the  arguments  which  precede  it.  The  case  of  an  optional  argument  not  being 
upplied  is  illustrated  below.  Note  the  general  form  of  this  fragment  shown  above. 

(Erange  1  10) 

same  as:  (S-apply  (S-frag  'Erange  ((state  UI)  (end  UI)  (by  A)  (Int  SO)) 

((setq  by  1)) 

((cond  ((>  state  end)  (done)))  (setq  Int  state)) 

((setq  state  (+  state  by))) 

0 

0) 

1 

10) 

I’hc  way  the  S-APPLY  forms  created  by  sequence  functions  arc  themselves  handled  is  discussed  below. 

Meta  Sequence  Functions 

1  .ike  sequence  functions,  meta  sequence  functions  produce  S-APPLYs  of  fragments.  However,  unlike  a 
equcncc  function,  some  of  the  arguments  to  a  meta  sequence  function  are  used  to  compute  what  the 
ragment  should  he.  The  men  sequence  function  REDUCES  is  used  as  an  example  in  the  pair  of  forms  below. 
'Jute  how  the  name  field  of  the  S-FRAG  is  used  to  record  the  initial  meta  sequence  function  expression. 

(Reduces  #'(  lambda  (a  b  c)  body )  inil  seql  seq2) 
same  as:  (S-apply  (S-frag  ’(Reduces  #’(lambda  (a  b  c)  body)  inil  seql  seq2) 

((In  UI)  (b  SI)  (C  SI)  (a  UO)) 

((setq  a  In)) 

((setq  a  body)) 

0 

0 

0) 

inil 

seql 

seq2) 

Note  that  since  the  variables  of  the  literal  LAMBDA  become  variables  of  the  resulting  fragment,  they  must 
ic  unique  in  body,  because  SUBST  may  be  used  to  rename  them.  If  a  literal  function  name  is  used  instead  of  a 
tend  LAMBDA,  then  it  is  converted  into  a  LAMBDA. 


Locating  Loop  Expressions 

Before  a  loop  expression  can  be  processed,  it  has  to  be  located  in  its  entirety.  Ilicrc  arc  three  ways  in 
,hich  this  can  happen,  the  easy  case  is  when  the  loop  is  delimited  by  a  LETS  or  DEFUNS.  In  that  ease  there  is 
o  difficulty  in  identifying  it 

Ihc  second  ease  is  also  quite  easy.  Whenever  any  of  the  sequence  functions  or  the  meta  sequence 
auctions  is  encountered  unexpectedly  (i.c..  not  during  the  processing  of  a  loop  which  has  already  been 
>cated)  it  is  wrapped  in  a  LETS.  Processing  then  continues  as  if  the  LETS  had  always  been  there. 

The  third  process  is  much  more  complex.  As  soon  as  a  loop  expression  is  located  by  cither  of  the  above 
ictJiods,  all  of  the  sequence  functions  and  meta  sequence  functions  inside  it  arc  expanded  into  S-APPLYs. 
my  LETScs  found  inside  are  processed  completely  as  subloops  before  processing  continues  on  the  outer  loop. 
Sole  that  no  other  macros  arc  expanded  at  this  stage  or  at  any  stage  during  the  processing  of  loop 
xpressions.)  Once  this  is  done  the  result  is  parsed  in  order  to  detect  nested  loops.  Once  they  arc  located, 
icy  arc  wrapped  in  LE  fSe>  and  immediately  processed. 
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Nested  loops  arc  located  by  looking  at  the  types  of  inputs  and  outputs  to  S-APPLYs  inside  a  loop 
expression.  Ihcsc  types  are  specified  in  die  argument  lists  of  the  fragments  the  S-APPLYs  are  applying.  To 
simplify  the  discussion  assume  for  the  moment  that  every  function  in  a  loop  expression  took  exactly  one 
argument,  l-.vcry  S-APPLY  tikes  in  cither  a  unitary  thing  U  or  a  sequence  S  and  returns  one  of  the  two.  They 
will  be  annotated  as  either  U-U,  U-S,  S-U.  or  S-S. 

Consider  the  examples  below  of  how  fragments  can  fit  together.  'Hie  first  example  shows  a  loop 
expression  where  everything  fits  together,  and  there  is  no  nested  loop.  The  second  example  shows  the 
prototypical  case  of  a  nested  loop.  If  the  two  inner  fragments  arc  grouped  together  into  a  subloop  and  then 
MAPSed  over  the  stream  coming  out  of  the  first  fragment,  then  everything  will  work  fine.  Otherwise  there  is 
no  good  way  to  make  tilings  fit  together.  The  search  for  nested  loops  focuses  on  finding  balanced 
subexpressions  which  clash  with  their  surroundings  at  both  ends.  The  only  real  difficulty  is  that  the 
expression  as  a  whole  may  he  unbalanced  at  either  end  (as  shown  in  the  final  two  examples)  so  there  is  no 
dependable  place  where  parsing 'can  start 

(Rsum  (Fgreater  (El  1st  X))) 

U-S  S-S  S-U 

(Rllst  (Rllst  (Ellst  (Ellst  x)))) 

U-S  U-S  S-U  S-U 

parsed  as:  U-S  <U-S  S-U>  S-U  1 

(Rllst  (Ellst  (Ellst  x))) 

U-S  S-U  S-U 

parsed  as:  <U-S  S-U>  S-U 

(Rllst  (Rllst  (Ellst  x))) 

U-S  U-S  S-U  , 

parsed  as:  U-S  <U-S  S-U> 

Note  that  ordinary  functions  embedded  in  loop  expressions  arc  MAPSed  if  they  end  up  receiving  a 
sequence  and  otherwise  are  just  executed  normally.  Therefore,  they  always  return  die  same  type  that  they 
receive  and  do  not  have  to  be  considered  when  looking  for  nested  loops.  The  real  parsing  algorithm  has  to 
work  on  tree-like  expressions  and  is  extended  accordingly.  It  looks  for  balanced  subtrees  which  clash  with  I 

their  surroundings  at  both  root  and  fringe.  They  arc  then  separated  out  as  subloops. 


LetS 

One  purpose  of  a  LE  TS  is  to  delineate  a  loop  as  discussed  above.  The  other  is  to  define  sequence  variables. 

All  of  the  variable  value  pairs  are  simplified  as  shown  below  by  putting  die  initializing  expressions  inside  the  ‘ 

LETS.  Note  that  this  means  that  these  expressions  cannot  refer  to  the  values  which  any  of  the  bound  variables 
have  outside  of  the  LETS.  Dcstructuring  is  handled  by  expanding  it  into  a  group  of  SETQs. 

(lets  ((x  (Erange  1  10)) 

((a  b)  (Ellst  list))) 

(reverse  (Rllst  (list  'Item  x  (+  a  b))))) 
becomes: (lets  (x  a  b  y) 

(setq  x  (Erange  1  10)) 

(setq  y  (El  1st  1 1st)) 

(setq  a  (car  y)) 

(setq  y  (edr  y)) 

(setq  b  (car  y))  < 

(reverse  (Rllst  (list  'Item  x  (+  a  b))))) 

Note  that  the  only  variables  which  carry  sequences  inside  the  body  of  the  LETS  arc  the  ones  specified  in 
the  bound  variable  list  All  of  the  free  variables  referred  to  in  the  body  are  unitary  no  matter  what  they  arc  in 
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the  place  where  they  arc  defined.  This  reflects  the  fact  that  if  this  loop  is  nested  in  another  loop  then  it  will  be 
MAPSed  and  so  any  sequences  in  that  loop  will  look  like  unitary  values  from  its  point  of  view.  Note  that  if 
sequences  were  multidimensional  objects  as  in  AIM.  then  things  would  be  much  more  complicated  because 
each  level  of  looping  would  only  strip  a  single  dimension  off  of  a  sequence. 


Implicit  MapS,  and  Coercions 

The  processing  of  the  body  of  a  LETS  starts  by  breaking  apart  all  of  the  S-APPLYs  as  follows.  SET  Os  of  new 
variables  arc  created  as  needed  so  that  every  argument  to  an  S-APPLY  is  a  variable,  and  the  output  of  every  S- 
APPLY  is  immediately  put  in  a  variable,  litis  transformation  is  illustrated  below.  Note  that  the  output 
variable  fields  of  the  S-APPLYs  arc  used  to  specify  the  destinations  for  their  outputs.  Ihc  use  of  a  MULTIPLE- 
VALUE  will  lead  to  a  list  of  more  than  one  variable  in  this  field,  lire  types  of  the  variables  arc  chosen  so  that 
the  types  match  the  argument  types  of  the  fragments  and  so  that  the  inputs  and  outputs  to  an  ordinary 
expression  arc  all  of  the  same  type.  The  fact  that  nested  loops  have  already  been  removed  guarantees  that  this 
is  in  fact  possible.  Note  that  execution  order  is  preserved  when  expressions  arc  broken  apart 

(lets  (x  a  b  y) 

(setq  x  (Erange  1  10)) 

(setq  y  (Elist  list)) 

(setq  a  (car  y)) 

(setq  y  (edr  y)) 

(setq  b  (car  y)) 

(reverse  (Rllst  (list  'Item  x  (+  a  b))))) 
becomes:(1et  (ul  u2  u3) 

(lets  (x  a  b  y  z) 

(setq  ul  1) 

(setq  u2  10) 

(S-apply  ( x )  (S-frag  ‘Erange  ...)  ul  uZ) 

(S-apply  (y)  (SJfrag  'Elist  ...)  Hat) 

(setq  a  (car  y)) 

(setq  y  (edr  y)) 

(setq  b  (car  y)) 

(setq  z  (list  'Item  x  (♦  a  b))) 

(S-apply  (u3)  (S-frag  'Rllst  ...)  z) 

(reverse  u3))) 
becomes: (let  (ul  u2  u3) 

(lets  (x  a  b  y  z) 

(S-r-vply  (ul)  (S-frag  '(at-start  #’{l«mbda  ()  1))  ...)) 

(S-apply  (uZ)  (S-frag  '(at-start  #’(1ambda  ()  10) )  ...)) 

(S-apply  (x)  (S-frag  'Erangt  ...)  ul  uZ) 

(S-apply  (y)  (S-frag  'Elist  ...)  Hat) 

(S-apply  (a)  (S-frag  '(mapS  #'car  ...)  ...)  y) 

(S-apply  (y)  (S-frag  '(mapS  f'cdr  ...)  ...)  y) 

(S-apply  (b)  (S-frag  ’(mapS  O' car  ...)  ...)  y) 

(S-apply  (z)  (S-frag  '(mapS  0' (lambda  (m  n  o) 

(list  m  (+  n  o) ) )  . . . )  . . . ) 

x  a  b) 

(S-apply  (u3)  (S-frag  'Rllst  ...>  z) 

(S-apply  T  (S-frag  ’(at-end  fravars#  ...)  ...)  u3))) 

While  things  are  being  decomposed,  the  following  coercions  are  applied.  Every  ordinary  expression  which 
was  nested  in  an  S-APPLY  which  requires  a  unitary  value  is  computed  AT-START.  F.vcry  ordinary  expression 
which  appeared  at  top  level  and  which  receives  unitary  output  values  from  sequence  functions  is  computed 
AT-END.  Every  other  ordinary  expression  is  MAPSed.  RLAST  is  used  to  coerce  the  value  of  the  last  form  to 
unitary  if  it  isn’t  unitary  already.  These  coercions  lead  to  the  third  version  of  the  LETS  above.  Note  that  the 
introduced  incta  sequence  functions  arc  immediately  expanded  into  S-APPLYs. 
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Combining  Fragments 

Once  all  of  the  appropriate  coercions  have  been  applied,  the  fragments  arc  combined  together  into  one 
large  fragment.  Ibis  is  done  in  two  stages,  First,  each  S-APPLY  is  converted  into  a  naked  fragment  Note 
that  every  argument  to  an  S-APPLY  is  a  variable,  and  its  outputs  go  directly  to  variables,  flbc  example  below 
shows  the  RUST  in  the  example  program  above.)  'ITie  S-APPLY  is  converted  into  a  naked  fragment  by  merely 
renaming  the  arguments  to  the  appropriate  variables  and  making  them  free  as  shown. 

(S-apply  (u3)  (S-frag  'Rllst  ((item  SI)  (result  UO)) 

((setq  result  nil)) 

((setq  result  (cons  item  result))) 

0 

((setq  result  (nreverse  result))) 

0) 

*) 

becomes: (S-frag  () 

((setq  u3  nil)) 

((setq  u3  (cons  z  u3))) 

0 

((setq  u3  (nreverse  u3))) 

0) 

Any  outputs  which  arc  not  used  arc  converted  to  aux  variables.  After  this  phase,  the  only  arguments 
which  remain  in  fragments  arc  aux  and  flag  variables.  The  outputs  of  the  last  form  in  the  LETS  arc  retained  as 
return  values  and  will  be  used  as  the  return  values  form  the  loop  as  a  whole.  Note  that  the  last  form  may  be  a 
VALUES. 

Now  that  everything  is  a  fragment,  they  are  all  combined  together  starting  at  the  top.  'Ibis  combination 
goes  pairwise  as  shown  below.  The  new  fragment  is  created  merely  by  concatenating  the  corresponding  parts 
of  the  two  initial  fragments.  As  a  result,  the  order  of  evaluation  is  preserved.  Note  that  due  to  all  of  the 
renaming  that  occurred  above,  all  of  the  data  (low  works  out  right  without  any  special  processing  being 
necessary.  Also  since  all  variable  names  in  the  original  fragments  were  GENSYNs  there  is  no  possibility  of 
unintentional  name  clashes. 

((S-FRAG  argsa  icodca  codeia  codeia  pcodea  ucodea ) 

(S-FRAG  argsb  icodeb  codelb  code2b  pcodeb  ucodeb )) 
becomes: (S-FRAG  argxa-argsb 
icodca-icodeb 
codela-codelb 
codein-codeib 
pcodea-pcodeb 
ucodea-ucodeb) 

Ibe  only  complexity  is  involved  with  filters.  If  any  of  the  variables  read  by  code  lb  or  code2b  are 
controlled  by  filter  flags  in  the  first  fragment,  then  both  codelb  and  code2b  arc  nested  in  S-IF  forms 
predicated  on  the  AND  of  these  flags.  For  example,  suppose  that  codelb  reads  two  sequence  variables  SI  and 
S2.  which  are  controlled  by  the  flags  FI  and  F2  respectively.  In  this  case,  codelb  would  be  converted  to 
(S-IF  (AND  FI  F2)  .  code  lb)  before  combination.  Code2b  would  be  converted  analogously. 
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The  Form  of  the  l-oops  Produced 

Once  all  of  the  fragments  have  been  combined  into  a  single  large  fragment,  this  fragment  is  converted  into 
a  loop  as  indicated  below.  'Ihc  various  parts  of  the  fragment  arc  merely  concatenated  together  into  the  body 
of  a  PROG.  Vur-list  is  a  list  of  all  of  the  aux,  flag,  and  return  variables  which  arc  specified  in  arg-lisi.  Ihc 
return-values  arc  the  return  variables  from  arg-list.  If  the  fragment  contains  any  ucade  then  the  PROG 
produced  is  wrapped  in  an  UNWIND- PROTECT  containing  this  ucode. 

(S-FRAG  args  icode  (T  code!  code2)  pcode ) 
becomes :( PROG  T  varlisl 
S  icode 
L  code! 
code2 
(90  L) 

E  pcode 

(RETURN-FftOM  T  .  return- values)) 

Note  that  the  PROG  produced  is  just  basic  Lisp.  (On  the  LispMachine  this  PROG  is  named  T  so  that  it  will 
be  transparent  to  the  user.)  'ITic  PROG  contains  a  number  of  variables  and  tags  created  by  the  macros.  These 
arc  all  GENSYMs  and  so  that  they  cannot  conflict  with  any  user  variables.  As  a  debugging  feature,  the  macros 
make  sure  that  all  of  the  stream  variables  specified  in  a  LETS  become  variables  in  the  PROG.  At  a  break  point, 
you  can  look  at  these  variables  in  order  to  see  the  current  element  in  each  of  the  corresponding  sequences. 
Also  for  debugging  convenience,  the  variable  LETS  :S- PROG  holds  the  PROG  produced  from  the  most  recently 
macro  expanded  loop  expression.  You  can  look  at  it  in  order  to  see  exactly  what  ccxlc  was  produced. 

The  form  (DONE)  expands  into  (GO  E).  The  form  (DONE  .  results )  expands  into  (RETURN,  results).  Note 
that  no  special  action  is  taken  with  regard  to  terminations,  they  just  end  up  in  (he  right  places  as  things  are 
combined  together.  The  form  (RESTART)  expands  into  (GO  S).  This  also  just  ends  up  in  the  right  place. 
Ihc  form  (S-IF  pred.  actions)  expands  into  (COND  (pred.  actions)). 

DefunS 

The  purpose  of  a  (DEFUNS  name  lambda-list .  body)  is  to  define  a  sequence  function.  The  body  is  exactly 
like  the  body  of  a  LETS.  In  addition  the  aux  variables  in  the  lambda-list  arc  just  like  LETS  variables.  These 
variables  and  the  body  arc  processed  exactly  as  described  above  in  order  to  create  a  fragment  The  arguments 
in  the  lambda-list  specify  that  some  of  the  free  variables  in  the  fragment  arc  actually  non-frec  inputs.  The 
fragment  is  modified  to  reflect  this.  Note  that  these  variables  must  lx;  unique  in  the  body  so  that  the  system 
can  use  SU8ST  to  rename  them.  A  sequence  function  macro  is  then  constructed  with  the  appropriate  name. 

Variable  Simplification 

One  problem  with  the  compilation  process  outlined  above  is  that  it  creates  a  vary  large  number  of 
variables  which  end  up  not  really  doing  anything  useful.  Due  to  the  fact  that  the  LispMachinc  compiler  is  not 
capable  of  optimizing  away  these  variables,  the  macro  package  performs  a  set  of  simplifications  in  order  to  get 
rid  of  them  itself. 

Ihc  following  simplifications  arc  performed  wherever  possible.  Note  that  this  process  is  applied  only  to 
the  variables  created  by  the  system.  The  variables  explicitly  declared  in  a  LETS  are  never  removed,  and  any 
free  variables  used  in  the  loop  arc  never  removed. 
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1)  If  3  variable  is  never  read  than  it  is  eliminated.  Any  computations  performed  to  assign  values  to  it 
arc  also  eliminated  if  it  can  be  established  that  there  is  no  possibility  of  a  side-effect  occurring 

2)  If  you  have  ( SET  0  X  Y)  and  X  is  SETQed  only  once  and  Y  is  not  SETQcd  in  the  range  of  reading  X, 
then  the  two  variables  can  be  merged  together  into  one  variable  eliminating  whichever  one  can  be 
eliminated. 

3)  If  you  have  (SETQ  X  EXPR)  and  X  is  read  only  once,  and  nothing  read  by  EXPR  is  modified  between 
here  and  the  use  of  x  and  there  is  no  possibility  of  trouble  with  side-effects  from  moving  EXPR,  then 
EXPR  can  be  substituted  for  X  eliminating  X. 

Another  area  where  needless  complexity  results  is  filters.  The  processing  above  leads  to  the  use  of  a 
number  of  flags  and  S- IF  forms.  These  arc  simplified  as  follows:  IftwoS-IFsin  a  row  arc  predicated  on  the 
same  flag  expression  then  they  arc  combined  together  into  one  in  order  to  reduce  the  number  of  references  to 
the  flags.  When  this  is  done  in  conjunction  with  the  variable  simplifications  above,  simple  eases  of  filters  end 
up  as  just  simple  CONOs. 


Functional  Summary 


-58- 


Watcre 


Appendix  B:  Functional  Summary 

This  Appendix  is  intended  as  a  short  reference  manual  for  the  system.  It  assumes  that  you  have  already 
read  the  rest  of  the  paper  and  just  gives  a  very  brief  description  of  each  of  the  macros  available  to  the  user. 
The  macros  arc  listed  in  logical  groupings.  Note  that  all  of  these  macro  names  arc  global  on  the  LispMachinc. 
The  summary  begins  with  a  description  of  the  basic  macros. 

latS  (( varvalue ) ...)  (rest  body 

'litis  has  two  purposes:  to  define  a  group  of  variables  which  contain  sequences  of  values,  and  to 
indicate  that  a  group  of  sequence  expressions  (the  Ixnly)  should  be  combined  together  into  a  single 
loop.  Fitch  value  will  be  coerced  to  a  sequence.  If  it  is  omitted  (or  if  the  var-valuc  pair  is  rendered  as 
merely  a  symbol)  then  the  initial  value  is  undefined  and  the  variable  must  be  written  before  it  can  be 
read.  A  tree  of  vans  instead  of  a  symbol  can  be  specified,  in  which  ease  dcstructuring  is  performed. 
Note  that  every  free  variable  is  per  force  unitary. 

All  of  the  expressions  in  the  body  arc  combined  into  a  single  loop.  Kach  unitary  expression  in  the 
body  will  be  automatically  MAPSed  if  possible.  'ITtc  only  time  it  is  not  possible  is  if  it  uses  the  output  of 
some  reducer.  In  this  latter  ease,  the  expression  will  he  automatically  computed  AT- END.  The  value  of 
the  last  expression  in  the  body  is  coerced  to  unitary  and  returned  as  the  value  of  the  loop. 

In  the  body,  you  can  use  SETQ  to. assign  to  a  sequence  variable,  multi  ple-value  can  be  used  to 
access  the  multiple  values  of  a  sequence  function.  The  last  form  can  be  a  VALUES  indicating  that 
multiple  values  arc  to  be  returned  from  the  loop  as  a  whole. 

defunS  name  lambda- list  (rest  body 

flic  purpose  of  this  form  is  to  define  a  new  sequence  function.  The  lambda-list  is  just  like  an  ordinary 
lambda  list  except  that  it  supports  only  the  following  four  keywords.  (UNITARY  indicates  that 
following  arguments  arc  unitary.  This  is  the  default  to  start  with.  (SEQUENCE  indicates  that  the 
following  arguments  carry  sequences.  (OPTIONAL  indicates  that  the  following  arguments  arc  optional. 
(AUX  indicates  that  die  following  arguments  arc  internal  variables.  With  both  of  the  last  two  cases 
default  values  can  be  specified  by  rendering  the  argument  as  a  variable-value  pair.  If  no  default  value 
is  specified  then  the  value  the  variable  is  undefined,  and  the  variable  must  be  written  before  it  can  be 
read. 

defuns  defines  a  macro  of  the  specified  name  defining  the  sequence  function  specified  by  body. 
The  lx)dy  is  exactly  like  the  body  of  a  LETS  except  that  it  is  not  immediately  coded  up  into  a  loop,  and 
the  value  of  the  last  expression  is  not  coerced  to  unitary.  Rather,  this  value  is  returned  whether  it  is 
unitary  or  a  sequence. 

don*  (rest  results 

In  a  loop  expression  the  macro  DONE  can  be  executed  in  order  to  indicate  that  the  loop  should  be 
immediately  terminated.  If  no  results  arc  specified.  then  the  loop  will  be  terminated  normally 
executing  all  AT-END  code,  and  returning  the  result  specified  by  the  List  expression.  If  any  result 
arguments  arc  supplied  then  they  will  be  returned  as  the  values  of  the  loop.  Note,  however,  that  in  this 
ease  any  AT-END  code  will  be  skipped.  Any  AT -UNWIND  code  is  executed  in  either  case. 

restart 

Kxccuting  this  inside  of  a  loop  expression  causes  the  immediately  containing  loop  to  be  restarted  at  the 
beginning.  All  of  the  loop  variables  arc  rcinitialired.  Typically,  some  side-effects  will  have  been 
performed  so  that  resuming  the  loop  will  lead  to  a  different  computation. 
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Meta  Sequence  Functions  ' 

The  meta  sequence  functions  take  in  ordinary  functions  and  convert  them  into  sequence  functions.  Hach 
one  takes  in  one  or  more  functional  arguments.  Hach  of  these  can  be  either  a  quoted  function  name,  or  a 
quoted  lambda  expression,  ora  macro  which  expands  into  either  one. 

maps  Junction  sequence... 

Ihc  nth  element  of  the  output  sequence  is  computed  by  applying  Junction  to  the  nth  elements  of  the 
input  sequences.  However,  if  the  nth  element  of  any  of  the  input  sequences  is  empty  then  function  is 
not  applied  and  the  nth  element  of  the  output  is  empty.  Note  that  the  length  of  the  output  sequence  is 
the  same  as  the  length  of  the  shortest  input  sequence, 
c.g.,  (mapS  #’+  [1  _  2  3  4]  [1  2  _  3])  ■>  [2 _ ft] 

scanS  function  init  sequence ... 

lit  is  is  just  like  MAPS  except  that  it  has  an  internal  state  variable.  'Hie  initial  (zeroth)  value  of  this 
variable  is  the  unitary  value  init.  The  elements  of  the  output  arc  the  successive  values  of  the  state  not 
including  its  zeroth  value.  The  nth  value  of  the  state  is  computed  by  calling  function  with  the  prior 
value  of  the  state  as  its  first  argument  and  the  nth  elements  of  the  inputs  as  its  remaining  arguments. 
However,  if  the  nth  element  of  any  of  the  input  sequences  is  empty  then  function  is  not  applied,  the 
state  is  not  changed,  and  the  nth  element  of  the  output  is  empty.  Ihc  length  of  the  output  sequence  is 
the  same  as  the  length  of  the  shortest  input  sequence. 
e.g..  ( scans  #'+0[l_23  4])  *>[1.3  810] 

f  1 1  terS  function  sequence ... 

Ihc  elements  of  the  output  sequence  are  computed  as  follows.  1  f  the  result  of  applying  Junction  to  the 
nth  elements  of  the  input  sequences  is  non-NIL  then  the  nth  element  of  the  first  input  is  used  as  the  nth 
element  of  the  output;  otherwise  the  nth  output  element  is  empty.  However,  if  the  nth  element  of  any 
of  the  input  sequences  is  empty  then  Junction  is  not  applied  and  the  nth  element  of  the  output  is 
empty.  Note  that  the  output  sequence  is  exactly  the  same  length  as  the  shortest  input  sequence; 
however,  some  of  the  output  sequence  slots  may  be  empty. 
c.g.,(f1  lterS r>  [1.2  3  4]  [0  2.3])  »>  [1 _ ] 

reduces  function  init  sequence ... 

Ibis  creates  a  sequence  function  with  an  internal  state  variable.  Ihc  state  is  initialized  to  the  (unitary) 
value  init.  ’Ihc  nth  value  of  the  state  is  computed  by  calling  function  with  the  prior  value  of  the  state  as 
its  first  argument  and  the  nth  elements  of  die  inputs  as  its  remaining  arguments.  However,  if  the  nth 
element  of  any  of  the  input  sequences  is  empty  then  Junction  is  not  applied  and  the  state  is  not 
changed.  When  the  input  sequences  arc  exhausted,  the  final  value  of  the  state  variable  is  returned  as 
the  (unitary)  result  If  there  arc  no  non-empty  elements  in  the  input  sequences  then  the  value  init  will 
be  returned.  This  form  is  equivalent  to:  (RLAST  (SCANS  function  init  sequence ...)  init). 
c.g.,  ( reduces  d '  +•  0  [12. 3]  [1.23  <])  ■>  8 
e.g.,  (reduces  d'  +  0  []  [1.2  3  4])  *>0 

generetoS  function  init  sequence... 

This  uses  an  internal  state  variable  in  order  to  generate  a  potentially  infinite  sequence  of  values.  The 
unitary  value  init  specifics  die  initial  (first)  value  of  the  state.  On  the  nth  cycle  of  the  loop,  function  is 
called  with  die  nth  value  of  the  state  as  its  first  argument  and  the  nth  elements  of  the  input  sequences 
(if  any)  as  its  remaining  arguments  in  order  to  compute  the  next  value  of  the  state.  I  lowcvcr,  if  the  nth 
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dement  of  any  of  the  input  sequences  is  empty  then  Junction  is  not  called  and  the  value  of  the  state  is 
not  changed.  The  output  sequence  consists  of  all  of  the  values  of  the  state  including  the  first  one  init. 
II' there  arc  no  input  sequences  (the  normal  case)  or  if  none  of  them  arc  futile,  then  the  output  will  be 
infinite.  If  any  of  the  input  sequences  is  finite,  then  the  length  of  the  output  will  be  the  same  the 
length  of  the  shortest  input.  Note  that  in  this  ease,  the  final  value  of  the  state  will  not  be  returned  as 
part  of  the  output 

c.g.,  (generates  »'  1+  0)  *>[0  1  23  4  56  7  . . .  ] 

c.g.,  (generates#' (lambda  (prev  new)  new)  NIL  [1  2  3  4])  *>  [NIL  12  3] 
truncates  function  sequence... 

This  is  used  to  create  sequence  functions  which  take  in  potentially  infinite  sequences  and  return 
sequences  which  have  been  truncated  to  finite  length.  I  he  function  argument  is  applied  to  successive 
groups  of  corresponding  elements  of  the  input  sequences.  The  output  sequence  is  composed  of  the 
elements  of  the  first  input  sequence  up  to  but  not  including  the  first  element  corresponding  to  a 
non-NiL  evaluation  of  function.  As  with  the  other  meta  sequence  functions,  if  any  of  the  nth  elements 
of  the  input  sequences  arc  empty  then  fiaiction  is  not  applied  and  the  nth  output  element  is  empty. 
Note  that  the  output  sequence  is  typically  shorter  than  any  of  the  input  sequences,  and  can  be  of  length 
zero. 

c.g.,  (truncates #'<  [1.2  3  4]  [0  2_4])  *>[1 _ ] 

e.g.,  ( truncates  #’>  [1  _  2  3  4]  [0  2  _  4])  «>  [] 

enumerates  truncate-function  generate-fiunction  init 

This  is  an  abbreviation  for  (TRUNCATES  truncate-function  (GENERATES  generate- function  init)).  It  is 

the  preferred  way  to  define  an  enumerator. 

c.g.  (enumerates #' zerop  #'l-.  5)  *>[54  3  2  1] 

at-  star  t  function  arg ... 

This  computes  ( function  arg ... )  in  the  initialization  code  before  a  loop  begins.  All  of  the  args  must  be 
unitary  values. 

at-end  Junction  arg... 

This  computes  (function  arg ...)  in  the  epilog  code  after  a  loop  ends.  All  of  the  args  must  be  unitary 
values.  They  can  be  values  returned  by  reducers.  Note  (hat  this  will  not  be  executed  if  the  loop  is 
terminated  via  a  DONE  with  arguments  or  by  some  extraordinary  exit  such  as  a  THROW. 

at- unwind  function  arg ... 

This  computes  (function arg ...)  in  an  UNWIND-PROTECT  wrapped  around  the  loop.  AH  of  the  args 
must  be  unitary  values.  They  can  be  values  returned  by  reducers,  ’lire  difference  between  this  and  AT- 
END  is  that  it  will  be  executed  no  matter  how  the  loop  is  terminated. 
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Pi,  Mined  Generators 

Gstquanca  arg 

This  takes  in  a  onimry  argument  and  produces  an  infinite  sequence  of  that  value.  Note  that  live 
successive  elements  of  the  sequence  will  all  be  EQ. 
e.g.,  (Gsequence  1)  *>  [1 1  1  . . .] 

Gprtvlous  sc^umr&optlonal  (first  NIL) 

This  Likes  in  a  sequence  and  returns  a  sequence  which  is  shifted  right  one  position.  First  is  used  as  the 
first  element  of  the  output,  and  the  last  element  of  the  input  is  discarded, 
c.g.,  (Gprevlous  [1  2  3  4]  0)  =>[0  12  3] 

G1 1st  list 

This  generates  the  successive  elements  of  list.  It  will  get  an  error  if  it  encounters  a  non-list  CDR. 

C.g.,  (G1 1st  ’(123))  =>[12  3  NIL  NIL  NIL  ...] 

Gsubl  Ista  list 

This  generates  the  successive  CDRs  of  list.  It  will  get  an  error  if  it  encounters  a  non-list  CDR. 
e.g..  (Gsubl  Ists  ’(1  2  3))  •>  [(1  2  3)  (2  3)  (3)  NIL  NIL  NIL  ...] 

Grange  ^optional  (first  l)  (step-size- 1) 

This  generates  fixnums  from  first  adding  step-size  at  each  step.  Note  that  step-size  can  be  negative. 
c.g.,  (Grange  10  2)  =>[10  12  14  ...] 

Predefined  Enumerators 

Eilat  list 

This  enumerates  the  successive  elements'of  list  up  to  and  not  including  the  first  NULL  sublisL  It  will  get 
an  error  if  it  encounters  a  non-list  CDR. 
e.g., (El  1st  '(123))  =>[123] 
e.g.,  (Eilat  nil )  ■>  [] 

Eaubl  lata  list 

This  enumerates  the  successive  CDRs  of  list  up  to  and  not  including  the  first  NULL  sublisL  It  will  get  an 

error  if  it  encounters  a  non-list  CDR. 

e.g..  (Esubllats  (1  2  3))  =>  [(1  2  3)  (2  3)  (3)] 

Eilat*  list 

This  enumerates  the  successive  elements  of  list  up  to  and  including  the  first  NULL  or  non-list  sublisL 
e.g.,  (Eilat*  '(12.  3))  «>  [l  2  3) 
e.g.,  (Eilat*  NIL)  »>  [NIL] 

Epl  lat  plist  »>  sequence- of- properties  sequence-ofi  values 

This  creates  two  sequence  outputs  consisting  of  the  successive  property  names  and  property  values 
respectively  of  the  naked  plist  plist.  Note  that  the  function  PLIS1  returns  the  CDR  of  a  naked  plist  not 
a  naked  plist 

C.g.,  (Epl  1st  '(NILA  102))  «>[A0][1  2] 
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E«1  1st  alisi  *>  sequencer  of- kns  sequcnce-ofi  values 

This  creates  two  sequences  as  outputs  consisting  of  the  successive  keys  and  values  respectively  of  alisi. 
It  requires  that  the  lists  of  values  associated  with  each  key  be  lists.  They  may  have  0. 1,  or  more  values 
in  them. 

c.g.,  (Eallst  *  ( ( A  1)  (B)  (C  2  3)))  »>  [ACC][123] 

E  r angt  first  Iasi  toptl  on  at  ( step-size  1 ) 

Creates  a  sequence  of  integers  by  counting  from  first  to  last  by  the  positive  increment  step-size. 
c.g.,  (Eranga  4  8  2)  ->[4  6  8] 

Evactor  vector  &op  t  Ion  a  1  (firstO)  (last  (l-  (array-length  vector))) 

This  enumerates  the  successive  elements  of  a  one  dimensional  array.  You  can  specify  a  subrange  of 
indices  by  specifying .//rtf  and  last.  (Note  that  this  will  not  work  on  MacLisp  arrays  of  numeric  type.) 
C.g.,  (Evector  <1  2  3>)  »>  [t  2  3] 

Eflle  filename 

This  creates  a  sequence  by  doing  successive  reads  on  the  file  until  end  of  file  is  reached.  File- name  can 
be  anything  acceptable  to  OPEN, 
c.g.,  (Efile  "data. lisp”)  ->[12  3] 
if  the  file  contains  "12  3" 


Predefined  Filters 

Fgreater  At’</i«we&opt1ona1  ( limit  0) 

This  takes  in  a  sequence  of  fixnums  and  restricts  it  to  a  sequence  containing  only  elements  greater  than 
limit. 

c.g.,  (Fgreater  [1  2  3]  2)  ->  [ _ 3] 

Predefined  Reducers 

Rlast  .wvp/cm-p&optlonal  (default  NIL) 

lliis  takes  in  a  sequence  and  returns  its  last  value.  If  the  sequence  has  zero  length  then  default  is 
returned. 

c.g.,  (Rlast  [I  2  3])  ■>  3 
c.g.,  (Rlast  []  NIL)  «>  NIL 

Rlgnore  sequence 

lliis  takes  in  a  sequence  and  returns  no  values  at  all.  It  is  useful  in  many  of  the  same  situations  as 

MAPC. 

c.g.,  (Rlgnora  [1  2  3])  ■> 

Rllst  sequence 

This  creates  a  list  of  the  elements  in  sequence.  The  order  of  the  dements  is  preserved, 
c.g.,  (Rllst  [12  3]) ->(12  3) 
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Rbag  sequence 

This  creates  a  list  of  the  elements  in  sequence.  The  order  of  the  elements  in  the  list  is  undefined.  This 
is  more  efficient  if  you  really  do  not  care  what  the  order  is.  ('ITic  order  ends  up  reversed,  but  you 
should  not  depend  on  that,  because  it  could  change  at  any  time.) 
c.g.,  (Rbag  Cl  2  3])  *>  (3  2  1) 

ill  1st*  sequence 

'Ihis  creates  a  list  of  the  elements  in  sequence  with  the  last  element  of  the  sequence  ending  up  as  tire 
COR  of  tire  last  CONS  cell  in  the  list 
c.g.,  (R1 1st.  [12  3])  «>  (12  .  3) 

C.g..  (R1 1st*  [1])  »>1 
C.g..  (Rllst*  [])  «>NIL 

Rnconc  sequence  ; 

'Ihis  creates  a  list  by  NCONCing  together  the  successive  elements  of  sequence.  'Ihis  is  what  MAPCAN  docs 
to  create  its  output 

C.g..  (Rnconc  [(1  2)  NIL  (3  4)])  ->(1  2  3  4) 

Rappend  sequence 

This  creates  a  list  by  APPGNDing  together  the  successive  elements  of  sequence. 
c.g.,  (Rappend  [(12)  NIL  (3  4)])  *>(1  2  3  4) 

Rset  sequence 

This  combines  the  elements  in  sequence  into  a  list  omitting  any  duplicate  elements.  'Ihc  order  of  this 
list  is  undefined.  Ihc  predicate  .which  is  used  to  test  for  duplicates  is  EQUAL, 
c.g.,  (Rset  [1  1(2)  (2)]).»>  ((2)  1) 

Reqset  sequence 

This  is  the  same  as  RSET  except  that  the  test  for  duplicates  is  EQ  instead  of  EQUAL. 

C.g.,  (Reqset  [1  1  (2)  (2)])  »>  ((2)  (2)  1) 

Rpl  1st  sequence-of-properiicsscquence-of-values 

Ihis  takes  in  a  sequence  of  property  names,  and  a  sequence  of  values  and  creates  a  naked  plist  Note 
that  the  function  SETPLIST  expects  to  receive  the  CDR  of  a  naked  plist  as  its  second  argument 
C.g.,  (Rpl  1st  [A  B]  [12])  ->  (NIL  B  2  A  1) 

Ral  1 1 1  sequence-of-keys  scquencc-of-values 

This  takes  in  a  sequence  of  keys,  and  a  sequence  of  values  and  creates  an  alist  All  of  the  values  which 
have  the  same  key  arc  combined  into  a  single  entry  in  the  alist  headed  by  the  key.  Ihc  predicate  which 
is  used  to  test  for  equality  of  keys  is  EQUAL. 

C.g.,  (Ral  1st  [(A)  8  (A)  8]  [1  2  3  4])  •>  ((B  4  2)  ((A)  3  1)) 

Raqal  1«t  scquence-of- keys  sequence- of- values 

This  is  identical  to  RAL  I  ST  except  that  the  test  for  key  equality  is  EQ. 

C.g.,  (Reqal  1st  [(A)  B  (A)  8]  [1  2  3  4])  •>  (((A)  3)  (8  4  2)  ((A)  l)) 
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Rvector  ir<  lor  sequence  &o  p  1 1  o  n  a  I  (J irstO )  (last  ( 1-  (array-length  vector))) 

'11)1$  takes  in  a  one  dimensional  array  and  a  sequence  of  elements  and  stores  those  elements  in 

successive  positions  in  the  array.  You  can  specify  a  specific  subrange  in  the  array.  (This  will  not  work 

with  Maclisp  arrays  of  numeric  type.)  Note  that  this  reducer  is  unusual  in  that  it  contains  a  terminator 

and  will  stop  the  loop  as  soon  as  the  vector  is  full 

C.g..  (Rvector  <NIL  NIL  NIL  NIL>  [1  2  3])  *><12  3  NIL> 

e.g..  (Rvector  <NIL  NIL>  [12  3])  *>  <1  2> 

Rf  lie  file- name  sequence 

lliis  takes  in  a  sequence  and  writes  all  of  its  elements  into  a  file.  File-  name  can  be  anything  acceptable 
to  OPEN. 

c.g..  (Rf lie  "data. lisp"  [1  2  3])  •>  T 

"<cr>l  <cr>2  <cr>3  "  Is  printed  In  "data.  1  isp" 

Rsum  scquence-of- integers 

Computes  the  sum  of  the  integers  in  its  input 
e.g.,  (Rsum  [1  2  3])  *>  8 

RsuiaS  sequence-of-jlonums 

Computes  the  sum  of  the  fionums  in  its  input 
C.g..  (RsumS[l.l  2.2  3.3])  *>8.8 

Rmax  sequence- of- numbers 

Computes  the  maximum  of  die  numbers  in  its  input  Returns  NIL  if  the  input  has  length  zero. 

C.g..  (Rmax  [1  2  3])  ->  3 
c.g..  (Rmax  [])*>  NIL 

Rrnln  scquencc-of-numbers 

Computes  the  minimum  of  the  numbers  in  its  input  Returns  NIL  if  die  input  has  length  zero, 
c.g..  (Rrnln  [1  2  3])  «>  1 
C.g..  (Rrnln  [])  *>  Nil 

Rcount  sequence 

Computes  the  number  of  elements  in  its  input 
e.g..  (Rcount  [1  2  3])  *>  3 

Rand  sequence 

Computes  the  AND  of  all  of  the  elements  of  sequence.  As  with  AND,  the  return  value  is  either  NIL  or  the 

last  element  of  the  input 

c.g..  (Rand  [1  2  3])  «>  3 

c.g..  (Rand  [1  NIL  2]) ->  NIL 

c.g..  (Rand  [])  *>  T 

Rand-fast  sequence 

This  is  the  same  as  RAND  except  that  the  loop  is  terminated  as  soon  as  a  NIL  value  (if  any)  is 
encountered. 

c.g..  (Rand-fast  [1  2  3])  «>  3  • 
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Computes  the  OR  of  all  of  the  elements  of  sequence.  As  with  OR,  the  return  value  is  either  NIL  or  the 

first  non-N  I L  element  of  the  input 

c.g..  (Ror  [12  3])  *>  1 

C.g..  (Ror  [NIL  NIL])  «■>  NIL 

C.g..  (Ror  [])  «>  NIL 

ist  sequence 

Phis  is  the  same  as  ROR  except  that  the  loop  is  terminated  as  soon  as  a  non-N  I L  value  (if  any)  is 
encountered. 

e.g..  (Ror-fast  [1  2  3])  ■>  1 


